Image processing apparatus and image processing method

ABSTRACT

The present disclosure relates to an image processing apparatus and an image processing method that are capable of improving coding efficiency at the time when a prediction using a correlation within a screen is performed. A predicted vector generation section sets, at a time of encoding of a current motion vector of a current block for a prediction using a correlation within a screen, a candidate block to not available and generates a predicted vector of the current motion vector by using a reference motion vector, the reference motion vector being referred to when the predicted vector of the current motion vector is generated. A difference vector generation section generates a difference vector between the current motion vector and the predicted vector generated by the predicted vector generation section. The present disclosure can be applied to, for example, an encoding apparatus.

CROSS REFERENCE TO PRIOR APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/516,992 (filed on Apr. 5, 2017), which is a National Stage PatentApplication of PCT International Patent Application No.PCT/JP2015/084763 (filed on Dec. 11, 2015) under 35 U.S.C. § 371, whichclaims priority to Japanese Patent Application No. 2014-265786 (filed onDec. 26, 2014), which are all hereby incorporated by reference in theirentirety.

TECHNICAL FIELD

The present disclosure relates to an image processing apparatus and animage processing method, and more particularly to, an image processingapparatus and an image processing method that are capable of improvingcoding efficiency at the time when a prediction using a correlationwithin a screen is performed.

BACKGROUND ART

There have been recently diffused apparatuses that handle imageinformation as digital and, at that time, for the purpose ofhighly-efficient information transmission and accumulation, employ acoding system for compression by orthogonal transformation and motioncompensation such as discrete cosine transform with use of redundancypeculiar to the image information, to perform compression coding onimages. Examples of this coding system include MPEG (Moving PictureExperts Group), H.264, and MPEG-4 Part10 (Advanced Video Coding,hereinafter described as AVC).

Now, for the purpose of higher improvement in coding efficiency thanH.264/AVC, standardization of a coding system called HEVC (HighEfficiency Video Coding) is being advanced by JCTVC (Joint CollaborationTeam-Video Coding), which is a joint standardization organization formedby ITU-T (International Telecommunication Union TelecommunicationStandardization Sector) and ISO/IEC (International Organization forStandardization/International Electrotechnical Commission).

Further, in HEVC, range extension (HEVC Range Extension) has beenstudied so as to support high-end formats, such as images in colordifference signal formats like 4:2:2 and 4:4:4, profiles for screencontent (see, for example, Non-Patent Literature 1).

Incidentally, Intra Block Copy (Intra BC) is a coding tool of performinga prediction by using a correlation within a screen and performingmotion compensation within the screen. Intra BC is known as a tool thatcontributes to improvement in coding efficiency for artificial imagessuch as computer screens and CG images.

However, the technique of Intra BC is not employed in theabove-mentioned HEVC Range Extension extension and is continued to bestudied for standardization of Screen Content Coding (SCC) extension(see, for example, Non-Patent Literature 2).

As of December 2014, in the standardization of the HEVC SCC extension,commonalizing Intra BC and inter coding is in discussion. In the casewhere Intra BC and inter coding are commonalized, it is necessary tocommunalize selection of a predicted vector (MV Predictor), transmissionof an index (RefIdx) of a reference picture, transmission of adifference vector (MVD) between a current motion vector and a predictedvector, and the like.

CITATION LIST Non-Patent Literature

-   Non-Patent Literature 1: Jill Boyce, et al. “Draft high efficiency    video coding (HEVC) version 2, combined format range extensions    (RExt), scalability (SHVC), and multi-view (MV-HEVC) extensions”,    JCTVC-R1013 v6, 2014.10.1-   Non-Patent Literature 2: Rajan Joshi, et al. “High Efficiency Video    Coding (HEVC) Screen Content Coding: Draft 1”, JCTVC-R1005-v3,    2014.9.27

DISCLOSURE OF INVENTION Technical Problem

However, in the case where a predicted vector generated with referenceto a motion vector used in inter coding is used at the time of encodingof a motion vector used in Intra BC, a difference vector becomes large,and coding efficiency is reduced.

The present disclosure has been made in view of the circumstances asdescribed above and is capable of improving coding efficiency at thetime when a prediction using a correlation within a screen is performed.

Solution to Problem

An image processing apparatus according to a first aspect of the presentdisclosure is an image processing apparatus including: a predictedvector generation section that sets, at a time of encoding of a currentmotion vector of a current block for a prediction using a correlationwithin a screen, in a case where a type of a reference picture of thecurrent block and a type of a reference picture of a candidate blockcorresponding to a candidate of a reference motion vector are differentfrom each other, the candidate block to not available and generates apredicted vector of the current motion vector by using the referencemotion vector, the reference motion vector being referred to when thepredicted vector of the current motion vector is generated; and adifference vector generation section that generates a difference vectorbetween the current motion vector and the predicted vector generated bythe predicted vector generation section.

An image processing method according to the first aspect of the presentdisclosure corresponds to the image processing apparatus according tothe first aspect of the present disclosure.

In the first aspect of the present disclosure, at the time of encodingof a current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the reference motion vector beingreferred to when a predicted vector of the current motion vector isgenerated, the candidate block is set to not available, the predictedvector of the current motion vector is generated by using the referencemotion vector, and a difference vector between the current motion vectorand the predicted vector is generated.

An image processing apparatus according to a second aspect of thepresent disclosure is an image processing apparatus including: apredicted vector generation section that sets, at a time of decoding ofa current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the candidate block to notavailable and generates a predicted vector of the current motion vectorby using the reference motion vector, the reference motion vector beingreferred to when the predicted vector of the current motion vector isgenerated; and

a motion vector generation section that adds a difference vector betweenthe current motion vector and the predicted vector to the predictedvector generated by the predicted vector generation section andgenerates the current motion vector.

An image processing method according to the second aspect of the presentdisclosure corresponds to the image processing apparatus according tothe second aspect of the present disclosure.

In the second aspect of the present disclosure, at the time of decodingof a current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the reference motion vector beingreferred to when a predicted vector of the current motion vector isgenerated, the candidate block is set to not available, the predictedvector of the current motion vector is generated by using the referencemotion vector, a difference vector between the current motion vector andthe predicted vector is added to the predicted vector, and the currentmotion vector is generated.

It should be noted that the image processing apparatuses according tothe first and second aspects can be achieved by causing a computer toexecute a program.

Further, the program executed by the computer in order to achieve theimage processing apparatuses according to the first and second aspectscan be provided by transmission via a transmission medium or recordingon a recording medium.

The image processing apparatuses according to the first and secondaspects may be independent apparatuses or may be internal blocks thatconfigure one apparatus.

Advantageous Effects of Invention

According to the first aspect of the present disclosure, an image can beencoded. Further, according to the first aspect of the presentdisclosure, it is possible to improve coding efficiency at the time whena prediction using a correlation within a screen is performed.

According to the second aspect of the present disclosure, an encodedstream can be decoded. Further, according to the second aspect of thepresent disclosure, it is possible to decode an encoded stream havingimproved coding efficiency at the time when a prediction using acorrelation within a screen is performed.

It should be noted that the effects described herein are not necessarilylimited and any one of the effects described herein may be produced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for describing Intra BC.

FIG. 2 is a diagram for describing a difference between Intra BC andinter coding.

FIG. 3 is a diagram for describing a type of a reference picture inHEVC.

FIG. 4 is a diagram showing an example of setting of a candidate blockin HEVC.

FIG. 5 is a diagram showing an example of setting of a candidate blockin HEVC.

FIG. 6 is a diagram for describing a candidate of a reference motionvector.

FIG. 7 is a diagram showing an example of setting of a candidate blockin a first embodiment.

FIG. 8 is a diagram showing an example of setting of a candidate blockin the first embodiment.

FIG. 9 is a diagram showing an example of a candidate of a referencepicture in HEVC.

FIG. 10 is a diagram for describing reference picture list creationprocessing in HEVC.

FIG. 11 is a diagram showing an example of a candidate of a referencepicture in the first embodiment.

FIG. 12 is a diagram for describing reference picture list creationprocessing in the first embodiment.

FIG. 13 is a block diagram showing a configuration example of a firstembodiment of an encoding apparatus to which the present disclosure isapplied.

FIG. 14 is a block diagram showing a configuration example of a codingsection of FIG. 13.

FIG. 15 is a flowchart for describing stream generation processing ofthe encoding apparatus of FIG. 13.

FIG. 16 is a flowchart for describing details of encoding processing ofFIG. 15.

FIG. 17 is a flowchart for describing details of the encoding processingof FIG. 15.

FIG. 18 is a flowchart for describing details of reference picturecreation processing of FIG. 16.

FIG. 19 is a flowchart for describing details of predicted vector listgeneration processing of FIG. 16.

FIG. 20 is a block diagram showing a configuration example of a firstembodiment of a decoding apparatus to which the present disclosure isapplied.

FIG. 21 is a block diagram showing a configuration example of a decodingsection of FIG. 20.

FIG. 22 is a flowchart for describing image generation processing of thedecoding apparatus of FIG. 20.

FIG. 23 is a flowchart for describing details of decoding processing ofFIG. 22.

FIG. 24 is a diagram for describing a predicted vector list generationmethod in a second embodiment.

FIG. 25 is a diagram for describing the predicted vector list generationmethod in the second embodiment.

FIG. 26 is a diagram for describing the predicted vector list generationmethod in the second embodiment.

FIG. 27 is a diagram showing an example of setting of a candidate blockin the second embodiment.

FIG. 28 is a flowchart for describing details of reference picturecreation processing in the second embodiment.

FIG. 29 is a flowchart for describing details of predicted vector listgeneration processing in the second embodiment.

FIG. 30 is a diagram for describing reference picture list creationprocessing in a third embodiment.

FIG. 31 is a flowchart for describing details of the reference picturelist creation processing in the third embodiment.

FIG. 32 is a block diagram showing a configuration example of computerhardware.

FIG. 33 is a diagram showing a schematic configuration example of atelevision apparatus to which the present disclosure is applied.

FIG. 34 is a diagram showing a schematic configuration example of amobile phone to which the present disclosure is applied.

FIG. 35 is a diagram showing a schematic configuration example of arecording/reproduction apparatus to which the present disclosure isapplied.

FIG. 36 is a diagram showing a schematic configuration example of animaging apparatus to which the present disclosure is applied.

FIG. 37 shows a schematic configuration example of a video set to whichthe present disclosure is applied.

FIG. 38 shows a schematic configuration example of a video processor towhich the present disclosure is applied.

FIG. 39 shows another schematic configuration example of the videoprocessor to which the present disclosure is applied.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, the premise of the present disclosure and modes forcarrying out the present disclosure (hereinafter, called embodiments)will be described. It should be noted that description will be given inthe following order.

0. Premise of the Present Disclosure (FIGS. 1 and 2)

1. First Embodiment: Encoding Apparatus and Decoding Apparatus (FIGS. 3to 23)

2. Second Embodiment: Encoding Apparatus and Decoding Apparatus (FIGS.24 to 29)

3. Third Embodiment: Encoding Apparatus and Decoding Apparatus (FIGS. 30and 31)

4. Fourth Embodiment: Computer (FIG. 32)

5. Fifth Embodiment: Television Apparatus (FIG. 33)

6. Sixth Embodiment: Mobile Phone (FIG. 34)

7. Seventh Embodiment: Recording/reproduction apparatus (FIG. 35)

8. Eighth Embodiment: Imaging Apparatus (FIG. 36)

9. Ninth Embodiment: Video Set (FIGS. 37 to 39)

<Premise of the Present Disclosure>

(Description on Intra BC)

FIG. 1 is a diagram for describing Intra Block Copy (Intra BC).

As shown in FIG. 1, in Intra BC, a reference picture used for predictinga PU (Prediction Unit) 11 within a picture 10 is the picture 10.Therefore, a block 12, which is included within the same picture 10 asthe PU 11 and has a high correlation with the PU 11, is set as areference block of the PU 11, and a vector 13 between the PU 11 and thePU 12 is detected as a motion vector.

(Description on Difference Between Intra BC and Inter Coding)

FIG. 2 is a diagram for describing a difference between Intra BC andinter coding.

As shown in FIG. 2, the precision of a motion vector in Intra BC isinteger-pixel precision, and the precision of a motion vector in intercoding is 1/4 pixel precision. Further, in Intra BC, since an encodedblock within the same screen is set as a reference block, at least oneof a length in a longitudinal direction and a length in a lateraldirection of the motion vector is longer. In inter coding, however,since an encoded block within a different screen is set as a referenceblock, the length of the motion vector depends on a motion between thescreens.

Further, in Intra BC, a picture before subjected to filter processing isused as a reference picture, but in inter coding, a picture aftersubjected to filter processing is used as a reference picture. Ingeneral, the picture before subjected to filter processing is stored ina cache, and the picture after subjected to filter processing is storedin a DRAM (Dynamic Random Access Memory).

As described above, the precision and the length of the motion vectorare different between Intra BC and inter coding. Therefore, if Intra BCand inter coding are communalized and a predicted vector generated withreference to the motion vector of inter coding is used at the time ofencoding of the motion vector of Intra BC, a difference vector becomeslarge and coding efficiency is reduced. It should be noted that themotion vector referred to when the predicted vector is generated ishereinafter called a reference motion vector.

Further, a destination to save the reference picture is differentbetween Intra BC and inter coding. Therefore, in the case where Intra BCand inter coding are commonalized, the cache and the DRAM are mixed asthe destinations to save the reference picture. However, since adecoding apparatus does not recognize its destination to save thereference picture, the decoding apparatus uselessly accesses the DRAM.As a result, the access bandwidth of the DRAM is increased.

First Embodiment

(General Outline of First Embodiment)

First, the general outline of a first embodiment will be described withreference to FIGS. 3 to 12.

FIG. 3 is a diagram for describing a type of a reference picture in HEVC(High Efficiency Video Coding).

As shown in FIG. 3, in HEVC, a type of a reference picture that has asmall difference in POC (Picture Order Count) from a picture to becurrently processed (hereinafter, called current picture) is set as STRP(Short-Term Reference Picture). Further, a type of a reference picturethat has a large difference in POC from the current picture (CurrentPic) is set as LTRP (Long-Term Reference Picture).

As shown in FIGS. 4 and 5, in the case where a reference picture of a PUto be currently processed (hereinafter, called current block) within thecurrent picture and a reference picture of a PU (hereinafter, candidateblock) corresponding to a candidate of a reference motion vector aredifferent in type from each other, the candidate block is set to notavailable.

For example, as shown in FIG. 3, in the case where the type of areference picture of a current block 21 is STRP, and the type of areference picture of a candidate block 22 corresponding to a candidate(PMV) of a reference motion vector of a current motion vector (targetMV) is LTRP, the candidate block 22 is set to not available. Thus, themotion vector of the candidate block 22 is excluded from the candidatesof the reference motion vector and is not used for generation of thepredicted vector.

It should be noted that candidate blocks are PUs that are adjacent tothe current block on the left, the upper left, the lower left, the top,and the upper right at the same time of day, and are PUs that arelocated at the same position as the current block and adjacent theretoon the lower right at a different time of day. Hereinafter, in the casewhere a candidate block at the same time of day as the current block isespecially distinguished from the other candidate blocks, the candidateblock is called a spatial-direction candidate block, and in the casewhere a candidate block at a time of day different from that of thecurrent block is especially distinguished from the other candidateblocks, the candidate block is called a temporal-direction candidateblock.

Further, as shown in FIGS. 4 and 5, in the case where the type of thereference picture of the current block and the type of the referencepicture of the candidate block are identical to each other, thecandidate block is set to available. Thus, the motion vector of thecandidate block is set to a candidate of the reference motion vector.

In the case where the motion vector of the candidate block when both thetype of the reference picture of the current block and the type of thereference picture of the candidate block are STRP is selected as areference motion vector, the predicted vector is generated by scaling ofthe reference motion vector on the basis of a difference in POC betweenthe reference picture of the current block and the reference picture ofthe candidate block.

Meanwhile, in the case where the motion vector of the candidate blockwhen both the type of the reference picture of the current block and thetype of the reference picture of the candidate block are LTRP isselected as a reference motion vector, the reference motion vector isset to a predicted vector without change (not scaled).

It should be noted that Intra BC and inter coding are not commonalizedin the present HEVC, and thus the type of a reference picture other thanLTRP and STRP is not assumed as shown in FIG. 5. Therefore, in the casewhere the type of a reference picture is “Intra BC”, which is the typeof a reference picture used in Intra BC, setting of a candidate block isundetermined.

In the case where Intra BC and inter coding are simply communalized, thetype of a reference picture used in Intra BC is different from an actualone and is set to STRP. Therefore, as shown in FIG. 6, a motion vector(PMV) of a candidate block 32, the actual type of a reference picture ofwhich is STRP, is included in candidates of a reference motion vector ofa current motion vector (target MV) used in Intra BC for a current block31.

As shown in FIG. 2, however, the precision and the length of the motionvector are different between Intra BC and inter coding. Therefore, ifthe motion vector of the candidate block 32 is selected as the referencemotion vector of the current motion vector of the current block 31, adifference vector between the current motion vector and a predictedvector becomes large. As a result, coding efficiency is reduced.

In this regard, in the first embodiment, the type of the referencepicture used in Intra BC, i.e., the type of the reference picture theactual type of which is “Intra BC”, is set to LTRP. Specifically, acurrent picture is newly defined as a candidate of the referencepicture, (If intra_block_copy_enabled_flag is equal to 1, the followingapplies. RefPicCurr=CurrPic). The current picture newly defined is thenadded as a candidate of the reference picture, the type of which is setto LTRP (All reference pictures that are included in RefPicCurr,RefPicSetLtCurr and RefPicSetLtFoll are marked as “used for long-termreference”).

Thus, the current picture is added to a reference picture list, which isa list of candidates of the reference picture, and is marked as “usedfor long-term reference”. As a result, setting of a candidate block withrespect to the current block the actual type of the reference picture ofwhich is “Intra BC” is performed similarly to setting of a candidateblock with respect to the current block the actual type of the referencepicture of which is LTRP.

Specifically, as shown in FIGS. 7 and 8, in the case where any one ofthe actual type of the reference picture of the current block and theactual type of the reference picture of the candidate block is “IntraBC” and the other one is STRP, the candidate block is set to notavailable. Thus, the motion vector of the candidate block is excludedfrom the candidates of the reference motion vector and is not used forgeneration of the predicted vector.

Further, as shown in FIGS. 7 and 8, in the case where both the actualtype of the reference picture of the current block and the actual typeof the reference picture of the candidate block are “Intra BC” or LTRP,the candidate block is set to available. Thus, the motion vector of thecandidate block is set to a candidate of the reference motion vector.

As described above, in the case where any one of the actual type of thereference picture of the current block and the actual type of thereference picture of the candidate block is “Intra BC” and the other oneis STRP, the motion vector of the candidate block is not used forgeneration of the predicted vector and coding efficiency is improved.

FIG. 9 is a diagram showing an example of a candidate of the referencepicture in HEVC.

As shown in FIG. 9, in HEVC, pictures (Short-term before Curr) that areprevious to a current picture (C) in display order and have a smalldifference in POC from the current picture are candidates of thereference picture. Further, pictures (Short-term after Curr) that aresubsequent to the current picture (C) in display order and have a smalldifference in POC from the current picture are candidates of thereference picture.

Moreover, pictures (Long-term) that are previous to the current picture(C) in display order and have a large difference in POC from the currentpicture are candidates of the reference picture. Further, pictures(Inter-layer) that are the same as the current picture (C) in displayorder but different in perspective, bit count, spatial resolution, S/N,frame rate, color difference signal format, and the like are candidatesof the reference picture.

FIG. 10 is a diagram for describing reference picture list creationprocessing for registering the candidates of the reference picture ofFIG. 9.

As shown in FIG. 10, in the reference picture list creation processing,first, an individual list in which information for identifying acandidate picture of the reference picture (hereinafter, called pictureidentifying information) is registered is created for each kind ofcandidates of the reference picture. In other words, aRefPicSetStCurrBefore list is created, in which the picture identifyinginformation of each of the pictures (Short-term before Curr) isregistered in ascending order of the difference in POC from the currentpicture.

Similarly, a RefPicSetStCurrAfter list of the pictures (Short-term afterCurr) and a RefPicSetLtCurr list of the pictures (Long-term) arecreated. Further, a RefPicSetIvCurr list is created, in which thepicture identifying information of each of the pictures (Inter-layer) isregistered in predetermined order.

For each piece of the picture identifying information registered in theRefPicSetStCurrBefore list, the RefPicSetStCurrAfter list, and theRefPicSetLtCurr list, used_by_curr can be set. The used_by_currindicates whether a picture identified by that picture identifyinginformation is used as a candidate of the reference picture of thecurrent picture. The used_by_curr is 1 when indicating that the pictureis used as a candidate of the reference picture of the current picture,and is 0 when indicating that the picture is not used.

In the example of FIG. 10, the used_by_curr of the head and secondpieces of the picture identifying information of theRefPicSetStCurrBefore list and the RefPicSetStCurrAfter list and thehead piece of the picture identifying information of the RefPicSetLtCurrlist are set to 1.

Next, on the basis of the individual lists, two temporal lists, i.e., aRefPicListTemp0 list that preferentially registers the candidates of thereference picture that are previous to the current picture in displayorder and a RefPicListTemp1 list that preferentially registers thecandidates of the reference picture that are subsequent to the currentpicture in display order, are created.

Specifically, the RefPicListTemp0 list is created, which registers inorder the pieces of the picture identifying information of theRefPicSetStCurrBefore list, for which the used_by_curr is set to 1, thepieces of the picture identifying information registered in theRefPicSetIvCurr list, the pieces of the picture identifying informationof the RefPicSetStCurrAfter list, for which the used_by_curr is set to1, and the piece of the picture identifying information of theRefPicSetLtCurr list, for which the used_by_curr is set to 1.

Further, the RefPicListTemp1 list is created, which registers in orderthe pieces of the picture identifying information of theRefPicSetStCurrAfter list, for which the used_by_curr is set to 1, thepieces of the picture identifying information of theRefPicSetStCurrBefore list, for which the used_by_curr is set to 1, thepiece of the picture identifying information of the RefPicSetLtCurrlist, for which the used_by_curr is set to 1, and the pieces of thepicture identifying information registered in the RefPicSetIvCurr list.

Next, the order of the pieces of the picture identifying informationregistered in the RefPicListTemp0 list and RefPicListTemp1 list ischanged as needed (Reference reordering).

Lastly, a reference picture list RefPicList0 is created, in which apredetermined number (five in the example of FIG. 10) of pieces of thepicture identifying information from the head, which are registered inthe RefpicListTemp0 list, are registered in order from the head.

Further, a reference picture list RefPicList1 is created, in which apredetermined number (four in the example of FIG. 10) of pieces of thepicture identifying information from the head, which are registered inthe RefpicListTemp1 list, are registered in order from the head.

It should be noted that a value obtained by subtracting 1 from thenumber of pieces of the picture identifying information registered inthe reference picture list RefPicList0 (RefPicList1) is disposed at aslice header or the like as num_ref_idx_10_active_minus1(num_ref_idx_11_active_minus1).

In the case where Intra BC and inter coding are commonalized, as shownin FIG. 11, it is necessary to add a current picture (CurrPic), which isthe reference picture in Intra BC, as a candidate of the referencepicture.

In the first embodiment, as described above, since the current pictureis defined as a candidate of the reference picture, a current picturelist (RefPicCurr) in which the picture identifying information of thecurrent picture is registered can be created in the reference picturelist creation processing. As a result, the picture identifyinginformation of the current picture is registered in the referencepicture list.

In the case where Intra BC and inter coding are simply communalized,however, it is unknown in which position of the reference picture listthe picture identifying information of the current picture isregistered.

Meanwhile, as shown in FIG. 2, the destination to save the referencepicture is different between Intra BC and inter coding. Therefore, inorder not to uselessly access the DRAM, when a decoding apparatusacquires an index of the picture identifying information of thereference picture within the reference picture list from an encodingapparatus, the decoding apparatus needs to recognize that the referencepicture is a reference picture in any of Intra BC and inter coding. Inother words, the decoding apparatus needs to recognize that thereference picture is any of the current picture and a picture differentfrom the current picture.

In this regard, in the first embodiment, as shown in FIG. 12, thepicture identifying information of the current picture, which isregistered in the current picture list, is registered in the heads ofthe two reference picture lists, the RefPicList0 list and theRefPicList1 list.

Thus, in the case where the decoding apparatus acquires the index in thehead of the reference picture list from the encoding apparatus, thedecoding apparatus can recognize that the reference picture is thecurrent picture, i.e., the reference picture in Intra BC. As a result,the decoding apparatus can read the current picture, as a referencepicture, from the cache without accessing the DRAM.

It should be noted that the picture identifying information of thecurrent picture may be registered in the RefPicList0 list only. Further,the position at which the picture identifying information of the currentpicture is registered can be set to an arbitrary predetermined positiondetermined in advance, such as the tail end, in addition to the head.

(Configuration Example of First Embodiment of Encoding Apparatus)

FIG. 13 is a block diagram showing a configuration example of a firstembodiment of an encoding apparatus as an image processing apparatus towhich the present disclosure is applied.

An encoding apparatus 50 of FIG. 13 includes a setting section 51, anencoding section 52, and a transmission section 53 and encodes a pictureby a system according to HEVC.

A unit of the encoding is a Coding UNIT (CU) having a recursivehierarchical structure. Specifically, the CU is set by dividing apicture into CTUs (Coding Tree Units) having a fixed size and dividingthe CTU into two by arbitrary times in the horizontal direction andvertical direction. The maximum size of the CU is an LCU (Largest CodingUnit), and the minimum size thereof is an SCU (Smallest Coding Unit).Further, the CU is divided into PUs or TUs (transform units).

The setting section 51 of the encoding apparatus 50 sets a parameter setsuch as an SPS (Sequence Parameter Set), a PPS (Picture Parameter Set),VUI (Video Usability Information), and SEI (Supplemental EnhancementInformation). The SPS includes information specifying the size of theLCU and the SCU, reference picture list information for creating areference picture list, and the like.

The reference picture list information is constituted of, for example,information specifying the picture identifying information registered inthe individual lists and the current picture list, used_by_curr, and thelike. The setting section 51 supplies the set parameter set to theencoding section 52.

Pictures in units of frame are input to the encoding section 52. Theencoding section 52 encodes the input pictures by a system according toHEVC. At that time, the parameter set supplied from the setting section51 is used as needed. The encoding section 52 generates an encodedstream from encoded data obtained as a result of encoding and theparameter set supplied from the setting section 51 and supplies theencoded stream to the transmission section 53.

The transmission section 53 transmits the encoded stream, which issupplied from the encoding section 52, to a decoding apparatus that willbe described later.

(Configuration Example of Encoding Section)

FIG. 14 is a block diagram showing a configuration example of theencoding section 52 of FIG. 13.

The encoding section 52 of FIG. 14 includes an A/D conversion section71, a screen sorting buffer 72, a computing section 73, an orthogonaltransformation section 74, a quantization section 75, a losslessencoding section 76, an accumulation buffer 77, a generation section 78,an inverse quantization section 79, an inverse orthogonal transformationsection 80, and an adder section 81. Further, the encoding section 52includes a filter 82, a frame memory 85, a switch 86, an intraprediction section 87, a list creation section 88, a motionprediction/compensation section 89, a predicted vector generationsection 90, a difference vector generation section 91, a predicted imageselection section 92, and a rate control section 93.

The A/D conversion section 71 of the encoding section 52 performs A/Dconversion on pictures in unit of frame, which are input as encodingtargets. The A/D conversion section 71 outputs the pictures, as digitalsignals after conversion, to the screen sorting buffer 72 to be storedtherein.

The screen sorting buffer 72 sorts the pictures in unit of frame, whichare stored in display order, in encoding order in accordance with theGOP (Group of Picture) structure. The screen sorting buffer 72 outputseach of the sorted pictures, as a current picture, to the computingsection 73, the intra prediction section 87, and the motionprediction/compensation section 89.

The computing section 73 performs encoding by subtracting a predictedimage, which is supplied from the predicted image selection section 92,from the current picture supplied from the screen sorting buffer 72. Thecomputing section 73 outputs the resultant picture, as residualinformation, to the orthogonal transformation section 74. It should benoted that in the case where the predicted image is not supplied fromthe predicted image selection section 92, the computing section 73outputs the current picture read from the screen sorting buffer 72, asresidual information, to the orthogonal transformation section 74without change.

The orthogonal transformation section 74 performs orthogonaltransformation on the residual information from the computing section 73in units of TU. The orthogonal transformation section 74 supplies anorthogonal transformation coefficient obtained as a result of theorthogonal transformation to the quantization section 75.

The quantization section 75 quantizes the orthogonal transformationcoefficient supplied from the orthogonal transformation section 74. Thequantization section 75 supplies the quantized orthogonal transformationcoefficient to the lossless encoding section 76.

The lossless encoding section 76 acquires intra prediction modeinformation, which indicates an optimal intra prediction mode, from theintra prediction section 87. Further, the lossless encoding section 76acquires inter prediction mode information, which indicates an optimalinter prediction mode, an index of the reference picture, and the likefrom the motion prediction/compensation section 89, and acquires motionvector information from the difference vector generation section 91.Moreover, the lossless encoding section 76 acquires offset filterinformation on adaptive offset filter processing from the filter 82.

The lossless encoding section 76 performs lossless encoding, such asvariable-length encoding (for example, CAVLC (Context-Adaptive VariableLength Coding) etc.) and arithmetic encoding (for example, CABAC(Context-Adaptive Binary Arithmetic Coding) etc.), on the quantizedorthogonal transformation coefficient supplied from the quantizationsection 75.

Further, the lossless encoding section 76 performs lossless encoding onthe intra prediction mode information or the inter prediction modeinformation, the motion vector information, the index of the referencepicture, the offset filter information, and the like, as encodinginformation on encoding. The lossless encoding section 76 disposes thelossless-encoded encoding information, a predetermined number(num_ref_idx_lo_minus1,num_ref_idx_lo_minus1) obtained by subtracting 1from a registration number of the reference picture list, and the likein the slice header or the like.

The lossless encoding section 76 adds the slice header to the orthogonaltransformation coefficient supplied from the quantization section 75 andthen supplies the resultant data, as encoded data, to the accumulationbuffer 77.

The accumulation buffer 77 temporarily stores the encoded data suppliedfrom the lossless encoding section 76. Further, the accumulation buffer77 supplies the stored encoded data to the generation section 78.

The generation section 78 generates an encoded stream from the parameterset supplied from the setting section 51 of FIG. 13 and the encoded datasupplied from the accumulation buffer 77 and supplies the encoded streamto the transmission section 53 of FIG. 13.

Further, the quantized orthogonal transformation coefficient output fromthe quantization section 75 is also input to the inverse quantizationsection 79. The inverse quantization section 79 inversely quantizes theorthogonal transformation coefficient, which is quantized by thequantization section 75, by a method corresponding to a quantizationmethod of the quantization section 75. The inverse quantization section79 supplies the orthogonal transformation coefficient obtained as aresult of inverse quantization to the inverse orthogonal transformationsection 80.

The inverse orthogonal transformation section 80 performs inverseorthogonal transformation on the orthogonal transformation coefficient,which is supplied from the inverse quantization section 79, in units ofTU by a method corresponding to an orthogonal transformation method ofthe orthogonal transformation section 74. The inverse orthogonaltransformation section 80 supplies the resultant residual information tothe adder section 81.

The adder section 81 adds the residual information supplied from theinverse orthogonal transformation section 80 and the predicted imagesupplied from the predicted image selection section 92, to locallydecode the current picture. It should be noted that in the case wherethe predicted image is not supplied from the predicted image selectionsection 92, the adder section 81 sets the residual information, which issupplied from the inverse orthogonal transformation section 80, to adecoding result. The adder section 81 supplies the locally-decodedcurrent picture to the frame memory 85. Further, the adder section 81supplies the current picture, all regions of which are decoded, as anencoded picture, to the filter 82.

The filter 82 performs filter processing on the encoded picture suppliedfrom the adder section 81. Specifically, the filter 82 performsdeblocking filter processing and adaptive offset filter (SAO (Sampleadaptive offset)) processing in sequence. The filter 82 supplies theencoded picture after subjected to the filter processing to the framememory 85. Further, the filter 82 supplies information indicating anoffset and a kind of the adaptive offset filter processing performed, asoffset filter information, to the lossless encoding section 76.

The frame memory 85 is constituted of, for example, a cache and a DRAM.The frame memory 85 stores the current picture, which is supplied fromthe adder section 81, in the cache and stores the encoded picture, whichis supplied from the filter 82, in the DRAM. Pixels adjacent to acurrent block in the current picture stored in the cache are set asperipheral pixels and supplied to the intra prediction section 87 viathe switch 86.

Further, the current picture stored in the cache and the encoded picturestored in the DRAM are set as reference pictures and output to themotion prediction/compensation section 89 via the switch 86.

The intra prediction section 87 performs intra prediction processing onthe current block in all candidate intra prediction modes by using theperipheral pixels read from the frame memory 85 via the switch 86.

Further, the intra prediction section 87 calculates a cost functionvalue (details thereof will be described later) for all candidate intraprediction modes on the basis of the current picture read from thescreen sorting buffer 72 and a predicted image generated as a result ofthe intra prediction processing. The intra prediction section 87 thendetermines an intra prediction mode, the cost function value of which isthe smallest, to be the optimal intra prediction mode.

The intra prediction section 87 supplies the predicted image, which isgenerated in the optimal intra prediction mode, and a corresponding costfunction value to the predicted image selection section 92. In the casewhere the intra prediction section 87 is notified of a selection of thepredicted image generated in the optimal intra prediction mode by thepredicted image selection section 92, the intra prediction section 87supplies the intra prediction mode information to the lossless encodingsection 76.

It should be noted that the cost function value is also called an RD(Rate Distortion) cost and is calculated on the basis of, for example, atechnique of a High Complexity mode or Low Complexity mode as determinedin JM (Joint Model), which is reference software in the H.264/AVCsystem. It should be noted that the reference software in the H.264/AVCsystem is publicly available inhttp://iphome.hhi.de/suehring/tml/index.htm.

Specifically, in the case where the High Complexity mode is employed asa technique of calculating the cost function value, all candidateprediction modes are temporarily subjected to processing to decoding,and a cost function value expressed by the following expression (1) iscalculated for each prediction mode.

[Mathematical Expression 1]

Cost(Mode)=D+λ·R  (1)

D represents a difference (distortion) between a raw image and a decodedimage, R represents the amount of generated codes including acoefficient of orthogonal transformation as well, and A represents aLagrange multiplier given as a function of a quantization parameter QP.

Meanwhile, in the case where the Low Complexity mode is employed as atechnique of calculating the cost function value, generation of apredicted image and calculation of the amount of codes of encodinginformation are performed for all candidate prediction modes, and a costfunction Cost (Mode) expressed by the following expression (2) iscalculated for each prediction mode.

[Mathematical Expression 2]

Cost(Mode)=D+QPtoQuant(QP)·Header_Bit  (2)

D represents a difference (distortion) between a raw image and apredicted image, Header_Bit represents the amount of codes of encodinginformation, and QPtoQuant represents a function given as a function ofa quantization parameter QP.

In the Low Complexity mode, it is only necessary to generate thepredicted images for all the prediction modes and unnecessary togenerate a decoded image. This leads to a smaller amount of computation.

The list creation section 88 performs the above-mentioned referencepicture list creation processing, for example, in units of slice, on thebasis of the reference picture list information included in the SPS setby the setting section 51 of FIG. 13, the information disposed in theslice header, and the like.

In other words, the list creation section 88 performs in sequencecreation of the individual lists, creation of the temporal lists,rearrangement of the order of the pieces of the picture identifyinginformation registered in the temporal lists, on the basis of thereference picture list information, the information disposed in theslice header, and the like, to create the reference picture list. Thelist creation section 88 then registers the picture identifyinginformation of the current picture in the head of the reference picturelist.

Further, the list creation section 88 sets SRTP or LTRP for each pieceof the picture identifying information registered in the referencepicture list, as a type of a reference picture of a picture identifiedby that picture identifying information. For example, the list creationsection 88 (setting section) sets LTRP (used for long-term reference),as a type of the current picture, for the picture identifyinginformation of the current picture. The list creation section 88 holdsthe reference picture list and also supplies the reference picture listto the motion prediction/compensation section 89.

The motion prediction/compensation section 89 performs motionprediction/compensation processing on the current block in all candidateinter prediction modes. Specifically, on the basis of the referencepicture list supplied from the list creation section 88, the motionprediction/compensation section 89 reads a picture, which is identifiedby picture identifying information registered in that reference picturelist, as a candidate of the reference picture, from the frame memory 85via the switch 86.

On the basis of the current picture from the screen sorting buffer 72and the current picture read as a candidate of the reference picture,the motion prediction/compensation section 89 detects in integer-pixelprecision a motion vector of Intra BC of the current block in allcandidate inter prediction modes. Further, on the basis of the currentpicture from the screen sorting buffer 72 and a picture other than thecurrent picture read as a candidate of the reference picture, the motionprediction/compensation section 89 detects in a fraction-pixel precisiona motion vector of inter coding of the current block in all candidateinter prediction modes.

On the basis of the detected motion vectors, the motionprediction/compensation section 89 then performs compensation processingon the candidates of the reference picture and generates a predictedimage. It should be noted that the inter prediction mode is a modeindicating the size of the current block and the like.

Further, the motion prediction/compensation section 89 calculates a costfunction value for all candidate inter prediction modes and thecandidates of the reference picture on the basis of the current picturesupplied from the screen sorting buffer 72 and the predicted image. Themotion prediction/compensation section 89 determines an inter predictionmode, the cost function value of which is the smallest, to be theoptimal inter prediction mode. The motion prediction/compensationsection 89 then supplies the smallest cost function value and acorresponding predicted image to the predicted image selection section92.

In the case where the motion prediction/compensation section 89 isnotified of a selection of the predicted image generated in the optimalinter prediction mode by the predicted image selection section 92, themotion prediction/compensation section 89 determines a motion vector ofthe current block corresponding to that predicted image to be a currentmotion vector and supplies the motion vector to the difference vectorgeneration section 91. Further, the motion prediction/compensationsection 89 determines a candidate of the reference picture correspondingto that predicted image to be a reference picture, and supplies an indexcorresponding to a position, in the reference picture list, of pictureidentifying information of that reference picture to the predictedvector generation section 90 and the lossless encoding section 76.Moreover, the motion prediction/compensation section 89 supplies theinter prediction mode information to the lossless encoding section 76.

The predicted vector generation section 90 generates a predicted vectorlist of the current motion vector for each reference picture list inwhich the reference picture is registered, on the basis of the indexsupplied from the motion prediction/compensation section 89.

Specifically, on the basis of the index, the predicted vector generationsection 90 reads a type of the reference picture corresponding to thatindex from the reference picture list held in the list creation section88, and holds the type. The predicted vector generation section 90 setsthe spatial-direction candidate block to a candidate block to beprocessed, in predetermined order, and sets the temporal-directioncandidate block to a candidate block to be processed, in predeterminedorder.

The predicted vector generation section 90 compares the type of thereference picture of the candidate block to be processed, which is heldtherein, and the type of the reference picture of the current block readfrom the list creation section 88. In the case where those types arematched with each other, the predicted vector generation section 90 setsthe candidate block to be processed to be available. In the case wherethose types are not matched with each other, the predicted vectorgeneration section 90 sets the candidate block to be processed to notavailable.

In the case where the candidate block to be processed is set toavailable, the predicted vector generation section 90 generates apredicted vector of the current motion vector by using a candidate ofthe reference motion vector that is a motion vector of the candidateblock to be processed. The predicted vector generation section 90 thengenerates a predicted vector list that resisters a predetermined numberof generated predicted vectors of the spatial-direction candidate blocksand temporal-direction candidate blocks. It should be noted that thenumber of generated predicted vectors does not reach a predeterminednumber, a zero vector or the like is registered in the predicted vectorlist. The predicted vector generation section 90 supplies the predictedvector list to the difference vector generation section 91.

The difference vector generation section 91 generates a differencevector between each of a predetermined number of predicted vectors,which are registered in the predicted vector list supplied from thepredicted vector generation section 90, and the current motion vector,to perform prediction encoding on the current motion vector. Thedifference vector generation section 91 supplies, the smallest value ofthe difference vector and an index corresponding to a position, in thepredicted vector list, of a corresponding predicted vector, to thelossless encoding section 76.

On the basis of the cost function values supplied from the intraprediction section 87 and the motion prediction/compensation section 89,the predicted image selection section 92 determines a mode having asmaller corresponding cost function value, in the optimal intraprediction mode and the optimal inter prediction mode, to be the optimalprediction mode. The predicted image selection section 92 then suppliesthe predicted image of the optimal prediction mode to the computingsection 73 and the adder section 81. Further, the predicted imageselection section 92 notifies the intra prediction section 87 or themotion prediction/compensation section 89 of a selection of thepredicted image of the optimal prediction mode.

The rate control section 93 controls the rate of a quantizationoperation of the quantization section 75 on the basis of the encodeddata accumulated in the accumulation buffer 77 such that an overflow orunderflow does not occur.

(Description on Processing of Encoding Apparatus)

FIG. 15 is a flowchart for describing stream generation processing ofthe encoding apparatus 50 of FIG. 13.

In Step S11 of FIG. 15, the setting section 51 of the encoding apparatus50 sets a parameter set. The setting section 51 supplies the setparameter set to the encoding section 52.

In Step S12, the encoding section 52 performs encoding processing ofencoding pictures in units of frame, which are input from the outside,by a system according to HEVC. Details of the encoding processing willbe described with reference to FIGS. 16 and 17 that will be describedlater.

In Step S13, the generation section 78 (FIG. 14) of the encoding section52 generates an encoded stream from the parameter set supplied from thesetting section 51 and accumulated encoded data and supplies the encodedstream to the transmission section 53.

In Step S14, the transmission section 53 transmits the encoded stream,which is supplied from the setting section 51, to a decoding apparatusthat will be described later, and then terminates the processing.

FIGS. 16 and 17 are flowcharts for describing details of the encodingprocessing in Step S12 of FIG. 15.

In Step S31 of FIG. 16, the A/D conversion section 71 of the encodingsection 52 performs A/D conversion on pictures in units of frame, whichare input as encoding targets, and outputs the pictures, as digitalsignal after conversion, to the screen sorting buffer 72 to be storedtherein.

In Step S32, the screen sorting buffer 72 sorts the pictures of frames,which are stored in display order, in encoding order in accordance withthe GOP structure. The screen sorting buffer 72 supplies each of thepictures in units of frame after the sorting, as a current picture, tothe computing section 73, the intra prediction section 87, and themotion prediction/compensation section 89.

In Step S33, the list creation section 88 performs reference picturelist creation processing, for example, in units of slice, on the basisof reference picture list information included in an SPS, informationdisposed in the slice header, and the like. Details of the referencepicture list creation processing will be described with reference toFIG. 18.

In Step S34, the intra prediction section 87 performs intra predictionprocessing on a current block in all candidate intra prediction modes byusing peripheral pixels read from the frame memory 85 via the switch 86.Further, the intra prediction section 87 calculates a cost functionvalue for all candidate intra prediction modes on the basis of thecurrent picture from the screen sorting buffer 72 and a predicted imagegenerated as a result of the intra prediction processing. The intraprediction section 87 then determines an intra prediction mode, the costfunction value of which is the smallest, to be the optimal intraprediction mode. The intra prediction section 87 supplies the predictedimage, which is generated in the optimal intra prediction mode, and acorresponding cost function value to the predicted image selectionsection 92.

Further, the motion prediction/compensation section 89 performs motionprediction/compensation processing on the current block in all candidateinter prediction modes on the basis of a reference picture list suppliedfrom the list creation section 88. Further, the motionprediction/compensation section 89 calculates a cost function value forall candidate inter prediction modes and the reference picture on thebasis of the current picture supplied from the screen sorting buffer 72and a predicted image generated as a result of the motionprediction/compensation processing. The motion prediction/compensationsection 89 determines an inter prediction mode, the cost function valueof which is the smallest, to be the optimal inter prediction mode. Themotion prediction/compensation section 89 then supplies the smallestcost function value and a corresponding predicted image to the predictedimage selection section 92.

In Step S35, the predicted image selection section 92 determines a modehaving the smallest cost function value, in the optimal intra predictionmode and the optimal inter prediction mode, to be the optimal predictionmode on the basis of the cost function values supplied from the intraprediction section 87 and the motion prediction/compensation section 89.The predicted image selection section 92 then supplies the predictedimage of the optimal prediction mode to the computing section 73 and theadder section 81.

In Step S36, the predicted image selection section 92 determines whetherthe optimal prediction mode is the optimal inter prediction mode or not.In the case where it is determined in Step S36 that the optimalprediction mode is the optimal inter prediction mode, the predictedimage selection section 92 notifies the motion prediction/compensationsection 89 of a selection of the predicted image generated in theoptimal inter prediction mode.

In accordance with this notification, the motion prediction/compensationsection 89 determines a motion vector of the current block correspondingto the predicted image to be a current motion vector and supplies thecurrent motion vector to the difference vector generation section 91.Further, the motion prediction/compensation section 89 determines acandidate of the reference picture corresponding to the predicted imageto be a reference picture and supplies an index corresponding to aposition, in the reference picture list, of picture identifyinginformation of that reference picture to the predicted vector generationsection 90.

In Step S37, the predicted vector generation section 90 performspredicted vector list generation processing of generating a predictedvector list of the current motion vector, on the basis of the indexsupplied from the motion prediction/compensation section 89. Details ofthe predicted vector list generation processing will be described laterwith reference to FIG. 19.

In Step S38, the difference vector generation section 91 generates adifference vector between each of a predetermined number of predictedvectors, which are registered in the predicted vector list supplied fromthe predicted vector generation section 90, and the current motionvector, to perform prediction encoding on the current motion vector. Thedifference vector generation section 91 then generates the smallestvalue of the difference vector and an index corresponding to a position,in the predicted vector list, of a corresponding predicted vector, asmotion vector information.

In Step S39, the motion prediction/compensation section 89 supplies theinter prediction mode information and the index of the reference pictureto the lossless encoding section 76, and the difference vectorgeneration section 91 supplies the motion vector information to thelossless encoding section 76. The processing then proceeds to Step S41.

Meanwhile, in the case where it is determined in Step S36 that theoptimal prediction mode is not the optimal inter prediction mode, thatis, in the case where the optimal prediction mode is the optimal intraprediction mode, the predicted image selection section 92 notifies theintra prediction section 87 of a selection of the predicted imagegenerated in the optimal intra prediction mode. In Step S40, the intraprediction section 87 supplies the intra prediction mode information tothe lossless encoding section 76, and advances the processing to StepS41.

In Step S41, the computing section 73 performs encoding by subtractingthe predicted image, which is supplied from the predicted imageselection section 92, from the current picture supplied from the screensorting buffer 72. The computing section 73 outputs the resultantpicture, as residual information, to the orthogonal transformationsection 74.

In Step S42, the orthogonal transformation section 74 performsorthogonal transformation on the residual information from the computingsection 73 in units of TU, and supplies a resultant orthogonaltransformation coefficient to the quantization section 75.

In Step S43, the quantization section 75 quantizes the orthogonaltransformation coefficient supplied from the orthogonal transformationsection 74 and supplies the quantized orthogonal transformationcoefficient to the lossless encoding section 76 and the inversequantization section 79.

In Step S44 of FIG. 17, the inverse quantization section 79 inverselyquantizes the quantized coefficient supplied the quantization section 75and supplies the resultant orthogonal transformation coefficient to theinverse orthogonal transformation section 80.

In Step S45, the inverse orthogonal transformation section 80 performsinverse orthogonal transformation on the orthogonal transformationcoefficient supplied from the inverse quantization section 79 in unitsof TU and supplies the resultant residual information to the addersection 81.

In Step S46, the adder section 81 adds the residual information suppliedfrom the inverse orthogonal transformation section 80 and the predictedimage supplied from the predicted image selection section 92, to locallydecode the current picture. The adder section 81 supplies thelocally-decoded current picture to the frame memory 85. Further, theadder section 81 supplies the current picture, all regions of which aredecoded, as an encoded picture, to the filter 82.

In Step S47, the filter 82 performs deblocking filter processing on theencoded picture supplied from the adder section 81.

In Step S48, the filter 82 performs adaptive offset filter processing onthe encoded picture after subjected to the deblocking filter processingfor each of the LCUs. The filter 82 supplies the resultant encodedpicture to the frame memory 85. Further, the filter 82 supplies offsetfilter information to the lossless encoding section 76 for each of theLCUs.

In Step S49, the frame memory 85 stores the current picture, which issupplied from the adder section 81, in the cache and stores the encodedpicture, which is supplied from the filter 82, in the DRAM. Pixelsadjacent to the current block in the current picture stored in the cacheare set as peripheral pixels and supplied to the intra predictionsection 87 via the switch 86. Further, the current picture stored in thecache and the encoded picture stored in the DRAM are set as referencepictures and output to the motion prediction/compensation section 89 viathe switch 86.

In Step S50, the lossless encoding section 76 performs lossless encodingon the intra prediction mode information or the inter prediction modeinformation, the motion vector information, the index of the referencepicture, and the offset filter information, as encoding information.

In Step S51, the lossless encoding section 76 performs lossless encodingon the quantized orthogonal transformation coefficient supplied from thequantization section 75. The lossless encoding section 76 then generatesencoded data from the encoding information lossless-encoded by theprocessing of Step S50 and the lossless-encoded orthogonaltransformation coefficient, and supplies the encoded data to theaccumulation buffer 77.

In Step S52, the accumulation buffer 77 temporarily accumulates theencoded data supplied from the lossless encoding section 76.

In Step S53, the rate control section 93 controls the rate of aquantization operation of the quantization section 75 on the basis ofthe encoded data accumulated in the accumulation buffer 77 such that anoverflow or underflow does not occur.

In Step S54, the accumulation buffer 77 outputs the stored encoded datato the generation section 78. The processing then returns to Step S12 ofFIG. 15 and proceeds to Step S13.

It should be noted that, for the purpose of simplifying description, theintra prediction processing and the motion prediction/compensationprocessing are constantly performed in the encoding processing of FIGS.16 and 17, but actually, any one of the processing is performeddepending on a picture type or the like in some cases.

FIG. 18 is a flowchart for describing details of the reference picturelist creation processing in Step S33 of FIG. 16.

In Step S71 of FIG. 18, the list creation section 88 creates fourindividual lists, i.e., a RefPicSetStCurrBefore list, aRefPicSetStCurrAfter list, a RefPicSetLtCurr list, and a RefPicSetIvCurrlist for each kind of candidates of the reference picture. In pictureidentifying information of each individual list, a type of a referencepicture of a picture identified by that picture identifying informationis set. Further, used_by_curr is set for each piece of the pictureidentifying information of the RefPicSetStCurrBefore list, theRefPicSetStCurrAfter list, and the RefPicSetLtCurr list.

In Step S72, the list creation section 88 creates a current picture listin which the picture identifying information of the current picture forwhich the LTRP is set as a type of the reference picture is registered.

In Step S73, the list creation section 88 creates two temporal lists,i.e., a RefpicListTemp0 list and a RefpicListTemp1 list on the basis ofthe picture identifying information of the individual list and theused_by_curr.

In Step S74, the list creation section 88 rearranges the order of thepieces of picture identifying information in the temporal lists, on thebasis of ref_pic_list_modification that is disposed in the slice headerand specifies the order of the pieces of picture identifying informationin the temporal lists.

In Step S75, on the basis of each of the temporal lists, the listcreation section 88 creates a reference picture list in which apredetermined number of pieces of picture identifying information fromthe head, which are registered in the temporal list, are registered inorder from the head.

In Step S76, the list creation section 88 registers the pictureidentifying information of the current picture, which is registered inthe current picture list, in the head of the reference picture list. Theprocessing then returns to Step S33 of FIG. 16 and proceeds to Step S34.

FIG. 19 is a flowchart for describing details of the predicted vectorlist generation processing in Step S37 of FIG. 16. The predicted vectorlist generation processing is performed for each of the referencepicture lists in which the reference picture is registered.

In Step S91 of FIG. 19, the predicted vector generation section 90determines a predetermined candidate block to be a candidate block to beprocessed. In the first processing of Step S91, a predeterminedspatial-direction candidate block is determined to be a candidate blockto be processed.

In Step S92, on the basis of the index supplied from the motionprediction/compensation section 89, the predicted vector generationsection 90 reads a type of the reference picture of the current blockcorresponding to that index from the reference picture list and holdsthe type. The predicted vector generation section 90 then determineswhether the read type of the reference picture of the current block isthe STRP or not.

In the case where it is determined in Step S92 that the type of thereference picture of the current block is the STRP, the processingproceeds to Step S93. In Step S93, the predicted vector generationsection 90 determines whether the type of the reference picture of thecandidate block to be processed, which is held therein, is the STRP ornot.

In the case where it is determined in Step S93 that the type of thereference picture of the candidate block to be processed is the STRP,that is, in the case where the type of the reference picture of thecurrent block and the type of the reference picture of the candidateblock are both the STRP, the processing proceeds to Step S94.

In Step S94, the predicted vector generation section 90 sets thecandidate block to be processed to available. In Step S95, on the basisof a difference in POC between the reference picture of the currentblock and the reference picture of the candidate block, the predictedvector generation section 90 performs scaling on the candidate of thereference motion vector, which is the motion vector of the candidateblock to be processed, and generates a predicted vector. The predictedvector generation section 90 then registers that predicted vector in thepredicted vector list and advances the processing to Step S100.

Meanwhile, in the case where it is determined in Step S92 that the typeof the reference picture of the current block is not the STRP, that is,in the case where the type of the reference picture of the current blockis the LTRP, the processing proceeds to Step S96.

In Step S96, similarly to the processing in Step S93, the predictedvector generation section 90 determines whether the type of thereference picture of the candidate block to be processed is the STRP ornot. In the case where it is determined in Step S96 that the type of thereference picture of the candidate block to be processed is the STRP,that is, in the case where the type of the reference picture of thecurrent block is the LTRP and the type of the reference picture of thecandidate block is the STRP, the processing proceeds to Step S97.

In Step S97, the predicted vector generation section 90 sets thecandidate block to be processed to not available and advances theprocessing to Step S100.

Meanwhile, in the case where it is determined in Step S93 that the typeof the reference picture of the candidate block to be processed is notthe STRP, that is, in the case where the type of the reference pictureof the current block is the STRP and the type of the reference pictureof the candidate block is the LTRP, the processing proceeds to Step S97.Thus, the candidate block to be processed is set to not available.

Further, in the case where it is determined in Step S96 that the type ofthe reference picture of the candidate block to be processed is not theSTRP, that is, in the case where the type of the reference picture ofthe current block and the type of the reference picture of the candidateblock are both the LTRP, the processing proceeds to Step S98.

In Step S98, the predicted vector generation section 90 sets thecandidate block to be processed to available. In Step S99, the predictedvector generation section 90 generates a candidate of the referencemotion vector, which is the motion vector of the candidate block to beprocessed, to be a predicted vector without change. The predicted vectorgeneration section 90 then registers that predicted vector in thepredicted vector list and advances the processing to Step S100.

In Step S100, the predicted vector generation section 90 determineswhether a predetermined number of predicted vectors are registered inthe predicted vector list or not. In the case where it is determined inStep S100 that a predetermined number of predicted vectors are not yetregistered, the processing returns to Step S91. A predeterminedspatial-direction candidate block or temporal-direction candidate block,which is not yet determined to be a candidate block to be processed, isdetermined to be a candidate block to be processed, and the subsequentprocessing is performed.

Meanwhile, in the case where it is determined in Step S100 that apredetermined number of predicted vectors are registered, the predictedvector generation section 90 supplies the predicted vector list to thedifference vector generation section 91. The processing then returns toStep S37 of FIG. 16 and proceeds to Step S38.

As described above, at the time of encoding of a current motion vectorused in Intra BC, in the case where the actual type of the referencepicture of the current block and the actual type of the referencepicture of the candidate block are different from each other, theencoding apparatus 50 sets the candidate block to not available. Thus,the motion vector of the candidate block, the actual type of thereference picture of which is different from the actual type of thereference picture of the current block, is not used for generation ofthe predicted vector. Therefore, the difference vector becomes small,and coding efficiency is improved.

Further, the encoding apparatus 50 sets the type of the referencepicture of the current block, the actual type of which is “Intra BC”, tothe LTRP. Thus, in the case where the actual type of the referencepicture of the candidate block is the STRP, the encoding apparatus 50sets the candidate block to not available. Therefore, it is possible toimprove coding efficiency in the case where Intra BC and inter codingare communalized, without largely changing HEVC in the case where IntraBC and inter coding are not communalized.

Moreover, the encoding apparatus 50 registers the picture identifyinginformation of the current picture in a predetermined position of thereference picture list of the current block. Therefore, a decodingapparatus that will be described later can know whether the referencepicture is a current picture or not on the basis of an indexcorresponding to a position, in the reference picture list, of thepicture identifying information of the reference picture. As a result,it is possible to prevent useless access to the DRAM from occurring inthe case where the reference picture is the current picture.

(Configuration Example of First Embodiment of Decoding Apparatus)

FIG. 20 is a block diagram showing a configuration example of a firstembodiment of a decoding apparatus as an image processing apparatus towhich the present disclosure is applied, the decoding apparatus decodingthe encoded stream transmitted from the encoding apparatus 50 of FIG.13.

The decoding apparatus 110 of FIG. 20 is constituted of a receptionsection 111, an extraction section 112, and a decoding section 113.

The reception section 111 of the decoding apparatus 110 receives theencoded stream transmitted from the encoding apparatus 50 of FIG. 13 andsupplies the encoded stream to the extraction section 112.

The extraction section 112 extracts the parameter set and the encodeddata from the encoded stream supplied from the reception section 111 andsupplies the parameter set and the encoded data to the decoding section113.

The decoding section 113 decodes the encoded data, which is suppliedfrom the extraction section 112, by a system according to HEVC. At thattime, the decoding section 113 also refers to the parameter set suppliedfrom the extraction section 112, as needed. The decoding section 113outputs a picture obtained as a result of the decoding.

(Configuration Example of Decoding Section)

FIG. 21 is a block diagram showing a configuration example of thedecoding section 113 of FIG. 20.

The decoding section 113 of FIG. 21 includes an accumulation buffer 131,a lossless decoding section 132, an inverse quantization section 133, aninverse orthogonal transformation section 134, an adder section 135, afilter 136, and a screen sorting buffer 139. Further, the decodingsection 113 includes a D/A conversion section 140, a frame memory 141, aswitch 142, an intra prediction section 143, a predicted vectorgeneration section 144, a motion vector generation section 145, a listcreation section 146, a motion compensation section 147, and a switch148.

The accumulation buffer 131 of the decoding section 113 receives theencoded data from the extraction section 112 of FIG. 20 and accumulatesthe encoded data. The accumulation buffer 131 supplies the accumulatedencoded data, as the encoded data of the current picture, to thelossless decoding section 132.

The lossless decoding section 132 performs lossless decoding, such asvariable-length decoding and arithmetic decoding, which corresponds tothe lossless encoding of the lossless encoding section 76 of FIG. 14, onthe encoded data from the accumulation buffer 131, to acquire thequantized orthogonal transformation coefficient and the encodinginformation. The lossless decoding section 132 supplies the quantizedorthogonal transformation coefficient to the inverse quantizationsection 133. Further, the lossless decoding section 132 supplies theintra prediction mode information and the like, as the encodinginformation, to the intra prediction section 143. The lossless decodingsection 132 supplies the index of the reference picture to the predictedvector generation section 144, supplies the motion vector information tothe motion vector generation section 145, and supplies the interprediction mode information and the index of the reference picture tothe motion compensation section 147.

Moreover, the lossless decoding section 132 supplies the intraprediction mode information or inter prediction mode information, as theencoding information, to the switch 148. The lossless decoding section132 supplies the offset filter information, as the encoding information,to the filter 136.

The inverse quantization section 133, the inverse orthogonaltransformation section 134, the adder section 135, the filter 136, theframe memory 141, the switch 142, the intra prediction section 143, thepredicted vector generation section 144, the list creation section 146,and the motion compensation section 147 respectively perform processingsimilar to the inverse quantization section 79, the inverse orthogonaltransformation section 80, the adder section 81, the filter 82, theframe memory 85, the switch 86, the intra prediction section 87, thepredicted vector generation section 90, the list creation section 88,and the motion prediction/compensation section 89. Thus, the picture isdecoded.

Specifically, the inverse quantization section 133 inversely quantizesthe quantized orthogonal transformation coefficient supplied from thelossless decoding section 132 and supplies the resultant orthogonaltransformation coefficient to the inverse orthogonal transformationsection 134.

The inverse orthogonal transformation section 134 performs inverseorthogonal transformation on the orthogonal transformation coefficientsupplied from the inverse quantization section 133 in units of TU. Theinverse orthogonal transformation section 134 supplies residualinformation, which is obtained as a result of the inverse orthogonaltransformation, to the adder section 135.

The adder section 135 adds the residual information supplied from theinverse orthogonal transformation section 134 and a predicted imagesupplied from the switch 148, to locally decode the current picture. Itshould be noted that in the case where the predicted image is notsupplied from the switch 148, the adder section 135 sets the residualinformation, which is supplied from the inverse orthogonaltransformation section 134, to a decoding result. The adder section 135supplies the locally-decoded current picture, which is obtained as aresult of the decoding, to the frame memory 141. Further, the addersection 135 supplies the current picture, all regions of which aredecoded, as a decoded picture, to the filter 136.

The filter 136 performs filter processing on the decoded picturesupplied from the adder section 135. Specifically, the filter 136 firstperforms deblocking filter processing on the decoded picture. Next, thefilter 136 performs adaptive offset filter processing, a kind of whichis indicated by the offset filter information, on the decoded pictureafter subjected to the deblocking filter processing for each of the LCUsby using an offset indicated by the offset filter information from thelossless decoding section 132. The filter 136 supplies the decodedpicture after subjected to the adaptive offset filter processing to theframe memory 141 and the screen sorting buffer 139.

The screen sorting buffer 139 stores, in units of frame, the decodedpictures supplied from the filter 136. The screen sorting buffer 139sorts the decoded pictures in units of frame, which are stored inencoding order, in the original display order, and supplies thosepictures to the D/A conversion section 140.

The D/A conversion section 140 performs D/A conversion on the decodedpictures in units of frame, which are supplied from the screen sortingbuffer 139, and outputs those pictures.

The frame memory 141 is constituted of, for example, a cache and a DRAM.The frame memory 141 stores the current picture, which is supplied fromthe adder section 135, in the cache and stores the decoded picture,which is supplied from the filter 136, in the DRAM. Pixels adjacent tothe current block in the current picture stored in the cache are set asperipheral pixels and supplied to the intra prediction section 143 viathe switch 142.

Further, the current picture stored in the cache or the decoded picturestored in the DRAM is set as a reference picture and output to themotion compensation section 147 via the switch 142.

The intra prediction section 143 performs the intra predictionprocessing of the optimal intra prediction mode, which is indicated bythe intra prediction mode information supplied from the losslessdecoding section 132, on the current block by using the peripheralpixels read from the frame memory 141 via the switch 142. The intraprediction section 143 supplies the resultant predicted image to theswitch 148.

The predicted vector generation section 144 generates a predicted vectorlist of the current motion vector on the basis of the index suppliedfrom the lossless decoding section 132 and the reference picture listheld in the list creation section 146, similarly to the predicted vectorgeneration section 90 of FIG. 14. The predicted vector generationsection 144 supplies the predicted vector list to the motion vectorgeneration section 145.

On the basis of the index in the motion vector information supplied fromthe lossless decoding section 132, the motion vector generation section145 reads a predicted vector corresponding to that index from thepredicted vector list supplied from the predicted vector generationsection 144. The motion vector generation section 145 adds the predictedvector and a difference vector in the motion vector information, toperform prediction decoding on the current motion vector. The motionvector generation section 145 supplies the resultant current motionvector to the motion compensation section 147.

Similarly to the list creation section 88 of FIG. 14, the list creationsection 146 performs the reference picture list creation processing onthe basis of the reference picture list information included in the SPSextracted by the extraction section 112 of FIG. 20, the informationdisposed in the slice header, and the like. The list creation section146 holds and supplies the resultant reference picture list to themotion compensation section 147.

The motion compensation section 147 performs motion compensationprocessing on the current block on the basis of the inter predictionmode information from the lossless decoding section 132, the index ofthe reference picture, and the current motion vector from the motionvector generation section 145.

Specifically, the motion compensation section 147 reads the pictureidentifying information of the index of the reference picture, in thepieces of the picture identifying information registered in the picturelist held in the list creation section 146. The motion compensationsection 147 reads a reference picture identified by the read pictureidentifying information from the frame memory 141 via the switch 142.The motion compensation section 147 performs motion compensationprocessing on the current block in the optimal inter prediction modeindicated by the inter prediction mode information, by using thereference picture and the current motion vector. The motion compensationsection 147 supplies the resultant predicted image to the switch 148.

In the case where the intra prediction mode information is supplied fromthe lossless decoding section 132, the switch 148 supplies the predictedimage, which is supplied from the intra prediction section 143, to theadder section 135. Meanwhile, in the case where the inter predictionmode information is supplied from the lossless decoding section 132, theswitch 148 supplies the predicted image, which is supplied from themotion compensation section 147, to the adder section 135.

(Description on Processing of Decoding Apparatus)

FIG. 22 is a flowchart for describing image generation processing of thedecoding apparatus 110 of FIG. 20.

In Step S111 of FIG. 22, the reception section 111 of the decodingapparatus 110 receives the encoded stream transmitted from the encodingapparatus 50 of FIG. 13 and supplies the encoded stream to theextraction section 112.

In Step S112, the extraction section 112 extracts the encoded data andthe parameter set from the encoded stream supplied from the receptionsection 111 and supplies the encoded data and the parameter set to thedecoding section 113.

In Step S113, the decoding section 113 performs decoding processing ofdecoding the encoded data, which is supplied from the extraction section112, by a system according to HEVC by using as needed the parameter setsupplied from the extraction section 112. Details of the decodingprocessing will be described with reference to FIG. 23 that will bedescribed later. The processing is thus terminated.

FIG. 23 is a flowchart for describing details of the decoding processingin Step S113 of FIG. 22.

In Step S131 of FIG. 23, the accumulation buffer 131 (FIG. 21) of thedecoding section 113 receives the encoded data in units of frame fromthe extraction section 112 and accumulates the encoded data. Theaccumulation buffer 131 supplies the accumulated encoded data, as theencoded data of the current picture, to the lossless decoding section132.

In Step S132, the lossless decoding section 132 performs losslessdecoding on the encoded data from the accumulation buffer 131 andacquires the quantized orthogonal transformation coefficient and theencoding information. The lossless decoding section 132 supplies thequantized orthogonal transformation coefficient to the inversequantization section 133.

Further, the lossless decoding section 132 supplies the intra predictionmode information and the like, as the encoding information, to the intraprediction section 143. The lossless decoding section 132 supplies theindex of the reference picture to the predicted vector generationsection 144, supplies the motion vector information to the motion vectorgeneration section 145, and supplies the inter prediction modeinformation and the index of the reference picture to the motioncompensation section 147.

Moreover, the lossless decoding section 132 supplies the intraprediction mode information or inter prediction mode information, as theencoding information, to the switch 148. The lossless decoding section132 supplies the offset filter information, as the encoding information,to the filter 136.

In Step S133, the list creation section 146 performs the referencepicture list creation processing, which is similar to the referencepicture creation processing of FIG. 18, on the basis of the referencepicture list information included in the SPS extracted by the extractionsection 112, the information disposed in the slice header, and the like.

In Step S134, the inverse quantization section 133 inversely quantizesthe quantized orthogonal transformation coefficient supplied from thelossless decoding section 132 and supplies the resultant orthogonaltransformation coefficient to the inverse orthogonal transformationsection 134.

In Step S135, the inverse orthogonal transformation section 134 performsinverse orthogonal transformation on the orthogonal transformationcoefficient supplied from the inverse quantization section 133 andsupplies the resultant residual information to the adder section 135.

In Step S136, the motion compensation section 147 determines whether theinter prediction mode information is supplied from the lossless decodingsection 132 or not. In the case where it is determined in Step S136 thatthe inter prediction mode information is supplied, the processingproceeds to Step S137.

In Step S137, the predicted vector generation section 144 performs thepredicted vector list generation processing, which is similar to thepredicted vector list generation processing of FIG. 19, on the basis ofthe index supplied from the lossless decoding section 132 and thereference picture list held in the list creation section 146.

In Step S138, on the basis of the index in the motion vectorinformation, the motion vector generation section 145 reads a predictedvector corresponding to that index from the predicted vector list. Themotion vector generation section 145 adds the predicted vector and adifference vector in the motion vector information, to performprediction decoding on the current motion vector. The motion vectorgeneration section 145 supplies the resultant current motion vector tothe motion compensation section 147.

In Step S139, the motion compensation section 147 performs the motioncompensation processing on the current block on the basis of the interprediction mode information and the index of the reference picture fromthe lossless decoding section 132, and the current motion vector fromthe motion vector generation section 145. The motion compensationsection 147 supplies the resultant predicted image to the switch 148 andadvances the processing to Step S141.

Meanwhile, in the case where it is determined in Step S136 that theinter prediction mode information is not supplied, that is, in the casewhere the intra prediction mode information is supplied to the intraprediction section 143, the processing proceeds to Step S140.

In Step S140, the intra prediction section 143 performs the intraprediction processing of the optimal intra prediction mode, which isindicated by the intra prediction mode information, on the current blockby using the peripheral pixels read from the frame memory 141 via theswitch 142. The intra prediction section 143 supplies a predicted imagegenerated as a result of the intra prediction processing to the addersection 135 via the switch 148 and advances the processing to Step S141.

In Step S141, the adder section 135 adds the residual informationsupplied from the inverse orthogonal transformation section 134 and thepredicted image supplied from the switch 148, to locally decode thecurrent picture. The adder section 135 supplies the locally-decodedcurrent picture, which is obtained as a result of the decoding, to theframe memory 141. Further, the adder section 135 supplies the currentpicture, all regions of which are decoded, as a decoded picture, to thefilter 136.

In Step S142, the filter 136 performs deblocking filter processing onthe decoded picture supplied from the adder section 135, to remove blockdistortion.

In Step S143, the filter 136 performs adaptive offset filter processingon the decoded picture after subjected to the deblocking filterprocessing for each of the LCUs, on the basis of the offset filterinformation supplied from the lossless decoding section 132. The filter136 supplies the picture after subjected to the adaptive offset filterprocessing to the screen sorting buffer 139 and the frame memory 141.

In Step S144, the frame memory 141 stores the current picture, which issupplied from the adder section 81, in the cache and stores the decodedpicture, which is supplied from the filter 136, in the DRAM. Pixelsadjacent to the current block in the current picture stored in the cacheare set as peripheral pixels and supplied to the intra predictionsection 143 via the switch 142. Further, the current picture stored inthe cache or the decoded picture stored in the DRAM is set as areference picture and output to the motion compensation section 147 viathe switch 142.

In Step S145, the screen sorting buffer 139 stores, in units of frame,the decoded pictures supplied from the filter 136, sorts the pictures inunits of frame, which are stored in encoding order, in the originaldisplay order, and supplies those pictures to the D/A conversion section140.

In Step S146, the D/A conversion section 140 performs D/A conversion onthe pictures in units of frame, which are supplied from the screensorting buffer 139, and outputs the pictures. The processing thenreturns to Step S113 of FIG. 22 and is terminated.

As described above, at the time of decoding of a current motion vectorused in Intra BC, in the case where the actual type of the referencepicture of the current block and the actual type of the referencepicture of the candidate block are different from each other, thedecoding apparatus 110 sets the candidate block to not available. Thus,the motion vector of the candidate block, the actual type of thereference picture of which is different from the actual type of thereference picture of the current block, is not used for generation ofthe predicted vector.

Therefore, the reference motion vector of the candidate block, theactual type of the reference picture of which is different from theactual type of the reference picture of the current block, is not usedfor generation of the predicted vector, and thus an encoded streamhaving improved coding efficiency can be decoded.

Further, the decoding apparatus 110 sets the type of the referencepicture of the current block, the actual type of which is “Intra BC”, tothe LTRP. Thus, in the case where the actual type of the referencepicture of the candidate block is the STRP, the decoding apparatus 110sets the candidate block to not available. Therefore, it is possible todecode an encoded stream having improved coding efficiency in the casewhere Intra BC and inter coding are communalized, without largelychanging HEVC in the case where Intra BC and inter coding are notcommunalized.

Moreover, the decoding apparatus 110 registers the picture identifyinginformation of the current picture in a predetermined position of thereference picture list of the current block. Therefore, the decodingapparatus 110 can know whether the reference picture is a currentpicture or not on the basis of an index corresponding to a position, inthe reference picture list, of the picture identifying information ofthe reference picture. As a result, it is possible to prevent uselessaccess to the DRAM from occurring in the case where the referencepicture is the current picture.

Second Embodiment

(General Outline of Second Embodiment)

The general outline of a second embodiment will first be described withreference to FIGS. 24 to 27.

In the second embodiment, the coding efficiency in the case where IntraBC and inter coding are commonalized is improved not by setting the typeof the current picture, which is the reference picture in Intra BC, tothe LTRP but by changing a method of generating a predicted vector listin the case where the actual type of the reference picture is “IntraBC”.

FIGS. 24 to 26 are diagrams for describing a method of generating apredicted vector list in the case where the actual type of the referencepicture is “Intra BC” in the second embodiment.

It should be noted that DiffPicOrderCnt (X, Y) represents a difference(X-Y) in POC between a picture X and a picture Y. Further,RefPicListX[refIdxLX] and LX[refIdxLX] (X=0 or X=1) each represent areference picture identified by picture identifying information of anindex refIdxLX of a reference picture of a current block, in pictureidentifying information registered in a reference picture listRefPicListX used for encoding of a slice including the current block.

Further, CurrPic represents a current picture, and ColPic represents apicture of a temporal-direction candidate block. Moreover, xNbA_(K) andyNbA_(K) respectively represent a position in an x direction and aposition in a y direction of a spatial-direction candidate block(hereinafter, called left candidate block) adjacent to the current blockon the left, the upper left, or the lower left. xNbB_(K) and yNbB_(K)respectively represent a position in the x direction and a position inthe y direction of a spatial-direction candidate block (hereinafter,upper candidate block) adjacent to the current block on the upper or theupper right.

listCol[refIdxCol] represents a reference picture identified by pictureidentifying information of an index refIdxCol of a reference picture ofa temporal-direction candidate block, in picture identifying informationregistered in a reference picture list RefPicListX used for encoding ofa slice including the temporal-direction candidate block.

As shown in FIG. 24, in the second embodiment, in the case where thecandidate block to be processed is the left candidate block, whenDiffPicOrderCnt (RefPicListX[refIdxLX], CurrPic)=0 and whenDiffPicOrderCnt(RefPicListX[refIdxLX], CurrPic) andDiffPicOrderCnt(RefPicListX[RefIdxLX [xNbA_(K)] [yNbA_(K)]], CurrPic)are equal to each other, the candidate block to be processed is set toavailable.

In other words, in the case where the reference picture of the currentpicture and the reference picture of the left candidate block are eachthe current picture, that is, in the case where the actual type of thereference picture of the current picture and the actual type of thereference picture of the left candidate block are “Intra BC”, the leftcandidate block is set to available. A candidate of a reference motionvector, which is a motion vector of the left candidate block, is thengenerated as a predicted vector without change.

Similarly, in the second embodiment, as shown in FIG. 25, in the casewhere the candidate block to be processed is the upper candidate block,when DiffPicOrderCnt(RefPicListX[refIdxLX], CurrPic)=0 and whenDiffPicOrderCnt(RefPicListX[refIdxLX], CurrPic) andDiffPicOrderCnt(RefPicListX[RefIdxLX [xNbB_(K)] [yNbB_(K)]], CurrPic)are equal to each other, the candidate block to be processed is set toavailable.

In other words, in the case where the actual type of the referencepicture of the current picture and the actual type of the referencepicture of the upper candidate block are “Intra BC”, the upper candidateblock is set to available. A candidate of a reference motion vector,which is a motion vector of the upper candidate block, is then generatedas a predicted vector without change. The generation of the predictedvector is performed for each of the reference picture lists in which thereference picture is registered.

Further, in the second embodiment, as shown in FIG. 26, in the casewhere the candidate block to be processed is the temporal-directioncandidate block, when DiffPicOrderCnt(LX[refIdxLX], CurrPic)=0 and whenDiffPicOrderCnt(LX[refIdxLX], CurrPic) andDiffPicOrderCnt(listCol[refIdxCol], ColPic) for colPb are equal to eachother, the candidate block to be processed is set to available.

In other words, in the case where the actual type of the referencepicture of the current picture and the actual type of the referencepicture of the temporal-direction candidate block are “Intra BC”, thetemporal-direction candidate block is set to available. A candidate of areference motion vector, which is a motion vector of thetemporal-direction candidate block, is then generated as a predictedvector without change.

As a result, as shown in FIG. 27, in the case where the actual type ofthe reference picture of the current picture is “Intra BC”, and only inthe case where the actual type of the reference picture of the candidateblock is “Intra BC”, the motion vector of the candidate block isgenerated as a predicted vector without change. Meanwhile, in the casesother than above, the candidate block is set to not available. It shouldbe noted that the type of the candidate of the reference picture, theactual type of which is “Intra BC”, is set to STRP.

Configurations of an encoding apparatus and a decoding apparatus in thesecond embodiment are similar to those of the encoding apparatus 50 ofFIG. 13 and the decoding apparatus 110 of FIG. 20, except for thereference picture list creation processing performed in the listcreation section 88 and the list creation section 146 and the predictedvector list generation processing performed in the predicted vectorgeneration section 90 and the predicted vector generation section 144.Therefore, hereinafter, the sections of the encoding apparatus 50 ofFIG. 13 and the decoding apparatus 110 of FIG. 20 will be used as thesections of the encoding apparatus and the decoding apparatus in thesecond embodiment, and only the reference picture list creationprocessing and the predicted vector generation processing will bedescribed.

(Description on Processing of Encoding Apparatus)

FIG. 28 is a flowchart for describing the reference picture listcreation processing of the list creation section 88 of the encodingapparatus 50 in the second embodiment.

In Step S161 of FIG. 28, the list creation section 88 creates fourindividual lists similarly to the processing of Step S71 in FIG. 18.STRP or LTRP is set for picture identifying information of each of theindividual lists.

In Step S162, the list creation section 88 creates a current picturelist in which the picture identifying information of the current picturefor which the STRP is set as a type of the reference picture isregistered.

Processing in Steps S163 to S166 are similar to the processing in StepsS73 to S76 of FIG. 18, and description thereof will thus be omitted.

It should be noted that, although not shown in the figures, thereference picture list creation processing of the list creation section146 of the decoding apparatus 110 in the second embodiment is alsoperformed similarly to the reference picture list creation processing ofFIG. 28.

FIG. 29 is a flowchart for describing details of the predicted vectorlist generation processing of the predicted vector generation section 90of the encoding apparatus 50 in the second embodiment. The predictedvector list generation processing is performed for each of the referencepicture lists in which the reference picture is registered.

Processing in Steps S181 to S183 of FIG. 29 are similar to theprocessing in Steps S91 to S93 of FIG. 19, and description thereof willthus be omitted. In the case where it is determined in Step S183 thatthe type of the reference picture of the candidate block to be processedis the STRP, the processing proceeds to Step S184.

In Step S184, the predicted vector generation section 90 determineswhether both the actual type of the reference picture of the currentblock and the actual type of the reference picture of the candidateblock are the STRP or not. Specifically, for example, the predictedvector generation section 90 determines whether a difference in POCbetween the reference picture of the current block and the currentpicture and a difference in POC between the reference picture of thecandidate block and the picture of the candidate block are both 1 ormore.

In the case where the predicted vector generation section 90 determinesthat the difference in POC between the reference picture of the currentblock and the current picture and the difference in POC between thereference picture of the candidate block and the picture of thecandidate block are both 1 or more, the predicted vector generationsection 90 determines that both the actual type of the reference pictureof the current block and the actual type of the reference picture of thecandidate block are the STRP, and advances the processing to Step S185.

Processing in Steps S185 and S186 are similar to the processing in StepsS94 and S95 of FIG. 19, and description thereof will thus be omitted.

Meanwhile, in the case where at least one of the difference in POCbetween the reference picture of the current block and the currentpicture and the difference in POC between the reference picture of thecandidate block and the picture of the candidate block is 0, thepredicted vector generation section 90 determines in Step S184 that atleast one of the actual type of the reference picture of the currentblock and the actual type of the reference picture of the candidateblock is not the STRP.

In Step S187, the predicted vector generation section 90 determineswhether both the actual type of the reference picture of the currentblock and the actual type of the reference picture of the candidateblock are “Intra BC” or not. Specifically, the predicted vectorgeneration section 90 determines whether the difference in POC betweenthe reference picture of the current block and the current picture andthe POC between the reference picture of the candidate block and thepicture of the candidate block are 0 or not.

In the case where the predicted vector generation section 90 determinesthat the difference in POC between the reference picture of the currentblock and the current picture and or the POC between the referencepicture of the candidate block and the picture of the candidate block isnot 0, the predicted vector generation section 90 determines that anyone of the actual type of the reference picture of the current block andthe actual type of the reference picture of the candidate block is not“Intra BC”, and advances the processing to Step S188.

In other words, in the case where the reference picture of the currentblock is the current picture but the picture of the candidate block andthe reference picture of the candidate block are different from eachother, and in the case where the picture of the candidate block and thereference picture of the candidate block are the same but the referencepicture of the current block is not the current picture, the predictedvector generation section 90 determines that the actual type of thereference picture of the current block and the actual type of thereference picture of the candidate block are different from each other,and advances the processing to Step S188.

In Step S188, the predicted vector generation section 90 sets thecandidate block to be processed to not available.

Further, in the case where it is determined in Step S182 that the typeof the reference picture of the current block is not the STRP, that is,in the case where the type of the reference picture of the current blockis the LTRP, the processing proceeds to Step S189.

In Step S189, similarly to the processing of Step S183, the predictedvector generation section 90 determines whether the type of thereference picture of the candidate block to be processed is the STRP ornot. In the case where it is determined in Step S189 that the type ofthe reference picture of the candidate block to be processed is theSTRP, that is, in the case where the type of the reference picture ofthe current block is the LTRP and the type of the reference picture ofthe candidate block is the STRP, the processing proceeds to Step S188.Thus, the candidate block to be processed is set to not available.

Meanwhile, in the case where it is determined in Step S189 that the typeof the reference picture of the candidate block to be processed is notthe STRP, that is, in the case where both the type of the referencepicture of the current block and the type of the reference picture ofthe candidate block are the LTRP, the processing proceeds to Step S190.Processing in Steps S190 and S191 are similar to the processing in StepsS98 and S99 of FIG. 19, and description thereof will thus be omitted.

Meanwhile, in the case where the predicted vector generation section 90determines that the difference in POC between the reference picture ofthe current block and the current picture and the difference in POCbetween the reference picture of the candidate block and the picture ofthe candidate block are both 0, the predicted vector generation section90 determines in Step S187 that both the actual type of the referencepicture of the current block and the actual type of the referencepicture of the candidate block are “Intra BC”.

The predicted vector generation section 90 then advances the processingto Step S190. Thus, the candidate block to be processed is set toavailable, and the candidate of the reference motion vector of thecandidate block to be processed is generated as a predicted vector.

After the processing of Step S186, S188, or S191, the processingproceeds to Step S192. The processing of Step S192 is similar to theprocessing of Step S100 in FIG. 19, and description thereof will thus beomitted.

It should be noted that, although not shown in the figures, thepredicted vector list generation processing of the predicted vectorgeneration section 144 of the decoding apparatus 110 in the secondembodiment is also performed similarly to the predicted vector listgeneration processing of FIG. 29.

Third Embodiment

(General Outline of Third Embodiment)

In a third embodiment, the picture identifying information of thecurrent picture is registered in the head of the temporal list, not inthe reference picture list.

In other words, in the third embodiment, as shown in FIG. 30, after thetwo temporal lists, i.e., the RefpicListTemp0 list and theRefpicListTemp1 list, are created on the basis of the individual listsof FIG. 10, the picture identifying information of the current pictureis registered in the head of each of the temporal lists.

Next, the order of the pieces of the picture identifying informationregistered in the RefpicListTemp0 list and the RefpicListTemp1 list ischanged as needed (Reference reordering). However, the order of thepicture identifying information of the current picture is prohibitedfrom being changed.

Lastly, a reference picture list RefpicList0 is created, in which thepicture identifying information in the head, and the pieces of thepicture identifying information in number (five in the example of FIG.30) obtained by adding 1 to num_ref_idx_10_active_minus1, which areregistered in the RefpicListTemp0 list, are registered in order from thehead.

Further, a reference picture list RefpicList1 is created, in which thepicture identifying information in the head, and the pieces of thepicture identifying information in number (four in the example of FIG.30) obtained by adding 1 to num_ref_idx_11_active_minus1, which areregistered in the RefpicListTemp1 list, are registered in order from thehead.

Configurations of an encoding apparatus and a decoding apparatus in thethird embodiment are similar to those of the encoding apparatus 50 ofFIG. 13 and the decoding apparatus 110 of FIG. 20, except for thereference picture list creation processing performed in the listcreation section 88 and the list creation section 146. Therefore,hereinafter, the sections of the encoding apparatus 50 of FIG. 13 andthe decoding apparatus 110 of FIG. 20 will be used as the sections ofthe encoding apparatus and the decoding apparatus in the thirdembodiment, and only the reference picture list creation processing willbe described.

(Description on Processing of Encoding Apparatus)

FIG. 31 is a flowchart for describing the reference picture listcreation processing of the list creation section 88 of the encodingapparatus 50 in the third embodiment.

Processing in Steps S201 to S203 of FIG. 31 are similar to theprocessing in Steps S71 to S73 of FIG. 18, and description thereof willthus be omitted.

In Step S204, the list creation section 88 registers the pictureidentifying information of the current picture, which is registered inthe current picture list, in the head of the temporal list.

In Step S205, the list creation section 88 rearranges the order of thepieces of picture identifying information in the temporal lists, otherthan the picture identifying information of the current picture, on thebasis of ref_pic_list_modification that is disposed in the slice header.

In Step S206, the list creation section 88 creates a reference picturelist on the basis of each of the temporal lists. In other words, thelist creation section 88 creates a reference picture list in which thepicture identifying information in the head, and the pieces of thepicture identifying information in number obtained by adding 1 tonum_ref_idx_10_active_minus1 (num_ref_idx_11_active_minus1) disposed inthe slice header, which are registered in the temporal list, areregistered in order from the head.

It should be noted that, although not shown in the figures, thereference picture list creation processing of the list creation section146 of the decoding apparatus 110 in the third embodiment is alsoperformed similarly to the reference picture list creation processing ofFIG. 31.

Fourth Embodiment

(Description on Computer to which Present Disclosure is Applied)

The series of processing described above can be executed by hardware orsoftware. In the case where the series of processing is executed bysoftware, a program configuring the software is installed in a computer.Here, the computer includes a computer incorporated in dedicatedhardware and, for example, a general-purpose personal computer that canexecute various functions by installing various programs therein.

FIG. 32 is a block diagram showing a hardware configuration example of acomputer that executes the series of processing described above by aprogram.

In the computer, a CPU (Central Processing Unit) 201, a ROM (Read OnlyMemory) 202, and a RAM (Random Access Memory) 203 are connected to oneanother via a bus 204.

Moreover, an input/output interface 205 is connected to the bus 204. Aninput section 206, an output section 207, a storage section 208, acommunication section 209, and a drive 210 are connected to theinput/output interface 205.

The input section 206 is constituted of a keyboard, a mouse, amicrophone, and the like. The output section 207 is constituted of adisplay, a speaker, and the like. The storage section 208 is constitutedof a hard disk, a nonvolatile memory, and the like. The communicationsection 209 is constituted of a network interface and the like. Thedrive 210 drives a removable medium 211 such as a magnetic disk, anoptical disc, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, the CPU 201 loads theprogram stored in, for example, the storage section 208 to the RAM 203via the input/output interface 205 and the bus 204 and executes theprogram, to perform the series of processing described above.

The program executed by the computer (CPU 201) can be provided by, forexample, being recorded on the removable medium 211 as a package mediumor the like. Further, the program can be provided via a wired orwireless transmission medium such as a local area network, the Internet,and digital satellite broadcasting.

In the computer, when the removable medium 211 is mounted to the drive210, the program can be installed in the storage section 208 via theinput/output interface 205. Further, the program can be received by thecommunication section 209 via the wired or wireless transmission mediumand installed in the storage section 208. In addition, the program canbe previously installed in the ROM 202 or the storage section 208.

It should be noted that the program executed by the computer may be aprogram that is processed chronologically along the described order inthis specification or may be a program that is processed in parallel orat a necessary timing such as when an invocation is performed.

Fifth Embodiment

(Configuration Example of Television Apparatus)

FIG. 33 exemplifies a schematic configuration of a television apparatusto which the present disclosure is applied. A television apparatus 900includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder904, a video signal processing section 905, a display section 906, anaudio signal processing section 907, a speaker 908, and an externalinterface section 909. Moreover, the television apparatus 900 includes acontrol section 910, a user interface section 911, and the like.

The tuner 902 selects a desired channel from broadcast wave signalsreceived by the antenna 901 for demodulation, and outputs an encoded bitstream obtained to the demultiplexer 903.

The demultiplexer 903 extracts a video or audio packet of a program tobe viewed from the encoded bit stream and outputs data of the extractedpacket to the decoder 904. Further, the demultiplexer 903 supplies apacket of data such as EPG (Electronic Program Guide) to the controlsection 910. It should be noted that in the case where scramble isperformed, the scramble is released by the demultiplexer or the like.

The decoder 904 performs packet decoding processing and outputs videodata generated by the decoding processing to the video signal processingsection 905 and audio data generated by the decoding processing to theaudio signal processing section 907.

The video signal processing section 905 performs noise removal, videoprocessing corresponding to user setting, or the like on the video data.The video signal processing section 905 generates video data of aprogram to be displayed on the display section 906, image data processedon the basis of an application supplied via a network, and the like.Further, the video signal processing section 905 generates video datafor displaying a menu screen to select items etc. and superimposes suchvideo data on the video data of the program. The video signal processingsection 905 generates a drive signal on the basis of the video data thusgenerated and drives the display section 906.

The display section 906 drives a display device (for example, liquidcrystal display element) on the basis of the drive signal from the videosignal processing section 905, to display a video of the program or thelike.

The audio signal processing section 907 performs predeterminedprocessing such as noise removal on the audio data, performs D/Aconversion processing or amplification processing on the processed audiodata, and supplies such audio data to the speaker 908, to perform audiooutput.

The external interface section 909 is an interface for connection withan external device or a network, and sends and receives data such as thevideo data and the audio data.

The user interface section 911 is connected to the control section 910.The user interface section 911 is constituted of an operation switch, aremote control signal reception section, and the like and supplies anoperation signal corresponding to a user operation to the controlsection 910.

The control section 910 is constituted of a CPU (Central ProcessingUnit), a memory, and the like. The memory stores programs executed bythe CPU, various types of data necessary for the CPU to performprocessing, EPG data, data acquired via the network, and the like. Theprograms stored in the memory are read and executed by the CPU at apredetermined timing such as activation of the television apparatus 900.The CPU executes the programs to control the sections such that thetelevision apparatus 900 operates according to the user operation.

It should be noted that in the television apparatus 900, a bus 912 isprovided to connect the tuner 902, the demultiplexer 903, the videosignal processing section 905, the audio signal processing section 907,the external interface section 909, and the like to the control section910.

In the television apparatus thus configured, the decoder 904 is providedwith a function of the decoding apparatus (decoding method) of thesubject application. This makes it possible to decode an encoded streamhaving improved coding efficiency at the time when a prediction using acorrelation within a screen is performed.

Sixth Embodiment

(Configuration Example of Mobile Phone)

FIG. 34 exemplifies a schematic configuration of a mobile phone to whichthe present disclosure is applied. A mobile phone 920 includes acommunication section 922, an audio codec 923, a camera section 926, animage processing section 927, a multiplex/demultiplex section 928, arecording/reproduction section 929, a display section 930, and a controlsection 931. Those components are connected to one another via a bus933.

Further, an antenna 921 is connected to the communication section 922,and a speaker 924 and a microphone 925 are connected to the audio codec923. Furthermore, an operation section 932 is connected to the controlsection 931.

The mobile phone 920 performs various operations such as sending andreception of audio signals, sending and reception of e-mail or imagedata, imaging, and data recording, in various modes such as a voicecommunication mode and a data communication mode.

In the voice communication mode, an audio signal generated in themicrophone 925 is subjected to conversion into audio data or datacompression in the audio codec 923 and then supplied to thecommunication section 922. The communication section 922 performsmodulation processing, frequency conversion processing, or the like onthe audio data to generate a sending signal. Further, the communicationsection 922 supplies the sending signal to the antenna 921 to be sent toa base station not shown in the figure. Further, the communicationsection 922 performs amplification, frequency conversion processing,demodulation processing, or the like on a received signal received inthe antenna 921 and supplies obtained audio data to the audio codec 923.The audio codec 923 performs data decompression of the audio data orconversion into an analog audio signal, and outputs the resultant signalto the speaker 924.

Further, in the case where mail is sent in the data communication mode,the control section 931 receives character data input by an operation ofthe operation section 932 and displays the input characters on thedisplay section 930. Further, the control section 931 generates maildata on the basis of a user instruction or the like in the operationsection 932 and supplies the mail data to the communication section 922.The communication section 922 performs modulation processing, frequencyconversion processing, or the like on the mail data and sends anobtained sending signal from the antenna 921. Further, the communicationsection 922 performs amplification, frequency conversion processing,demodulation processing, or the like on a received signal received inthe antenna 921, to restore mail data. The mail data is supplied to thedisplay section 930 to display mail content.

It should be noted that the mobile phone 920 also enables therecording/reproduction section 929 to store the received mail data in astorage medium. The storage medium is an arbitrary rewritable storagemedium. For example, the storage medium is a semiconductor memory suchas a RAM and a built-in flash memory, a hard disk, or a removable mediumsuch as a magnetic disk, a magneto-optical disk, an optical disc, a USB(Universal Serial Bus) memory, and a memory card.

In the case where image data is sent in the data communication mode,image data generated in the camera section 926 is supplied to the imageprocessing section 927. The image processing section 927 performsencoding processing on the image data and generates encoded data.

The multiplex/demultiplex section 928 multiplexes the encoded datagenerated in the image processing section 927 and audio data suppliedfrom the audio codec 923 by a predetermined system and supplies theresultant data to the communication section 922. The communicationsection 922 performs modulation processing, frequency conversionprocessing, or the like on the multiplexed data and sends an obtainedsending signal from the antenna 921. Further, the communication section922 performs amplification, frequency conversion processing,demodulation processing, or the like on a received signal received inthe antenna 921, to restore the multiplexed data. The multiplexed datais supplied to the multiplex/demultiplex section 928. Themultiplex/demultiplex section 928 demultiplexes the multiplexed data andsupplies encoded data to the image processing section 927 and audio datato the audio codec 923. The image processing section 927 performsdecoding processing on the encoded data to generate image data. Theimage data is supplied to the display section 930 to display a receivedimage. The audio codec 923 converts the audio data into an analog audiosignal and supplies the analog audio signal to the speaker 924, tooutput a received audio.

In the mobile phone device thus configured, the image processing section927 is provided with a function of the encoding apparatus and thedecoding apparatus (encoding method and decoding method) of the subjectapplication. This makes it possible to improve coding efficiency at thetime when a prediction using a correlation within a screen is performed.Further, it is possible to decode an encoded stream having improvedcoding efficiency at the time when a prediction using a correlationwithin a screen is performed.

Seventh Embodiment

(Configuration Example of Recording/Reproduction Apparatus)

FIG. 35 exemplifies a schematic configuration of arecording/reproduction apparatus to which the present disclosure isapplied. A recording/reproduction apparatus 940 records, for example,audio data and video data of a received broadcast program on a recordingmedium and provides the recorded data to a user at a timingcorresponding to a user instruction. Further, the recording/reproductionapparatus 940 can also acquire, for example, audio data and video datafrom another apparatus and record those pieces of data on a recordingmedium. Moreover, the recording/reproduction apparatus 940 decodes andoutputs the audio data and video data recorded on the recording medium,to enable a monitoring apparatus or the like to perform image display oraudio output.

The recording/reproduction apparatus 940 includes a tuner 941, anexternal interface section 942, an encoder 943, an HDD (Hard Disk Drive)section 944, a disk drive 945, a selector 946, a decoder 947, an OSD(On-Screen Display) section 948, a control section 949, and a userinterface section 950.

The tuner 941 selects a desired channel from broadcast wave signalsreceived by an antenna not shown in the figure. The tuner 941 outputs anencoded bit stream, which is obtained by demodulation of a receivedsignal of the desired channel, to the selector 946.

The external interface section 942 is constituted of at least any one ofan IEEE1394 interface, a network interface section, a USB interface, aflash memory interface, and the like. The external interface section 942is an interface for connection with an external device, a network, amemory card, and the like and receives data such as video data or audiodata to be recorded.

When the video data or audio data supplied from the external interfacesection 942 is not encoded, the encoder 943 performs encoding by apredetermined system and outputs an encoded bit stream to the selector946.

The HDD section 944 records content data such as video and audio,various programs, and other data in a built-in hard disk and reads thosepieces of data from the hard disk at the time of reproduction or thelike.

The disk drive 945 records and reproduces a signal with respect to amounted optical disc. The optical disc is, for example, a DVD disc(DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD+R, DVD+RW, etc.) or a Blu-ray(registered trademark) disc.

At the time of recording of video or audio, the selector 946 selects anyencoded bit stream from the tuner 941 or the encoder 943 and suppliesthe encoded bit stream to any of the HDD section 944 and the disk drive945. Further, at the time of reproduction of video or audio, theselector 946 supplies an encoded bit stream output from the HDD section944 or the disk drive 945 to the decoder 947.

The decoder 947 performs decoding processing on the encoded bit stream.The decoder 947 supplies video data generated by the decoding processingto the OSD section 948. Further, the decoder 947 outputs audio datagenerated by the decoding processing.

The OSD section 948 generates video data for displaying a menu screen toselect items etc. and superimposes such video data on the video dataoutput from the decoder 947 to be output.

The user interface section 950 is connected to the control section 949.The user interface section 950 is constituted of an operation switch, aremote control signal reception section, and the like and supplies anoperation signal corresponding to a user operation to the controlsection 949.

The control section 949 is constituted of a CPU, a memory, and the like.The memory stores programs executed by the CPU and various types of datanecessary for the CPU to perform processing. The programs stored in thememory are read and executed by the CPU at a predetermined timing suchas activation of the recording/reproduction apparatus 940. The CPUexecutes the programs to control the sections such that therecording/reproduction apparatus 940 operates according to the useroperation.

In the recording/reproduction apparatus thus configured, the encoder 943is provided with a function of the encoding apparatus (encoding method)of the subject application, and the decoder 947 is provided with afunction of the decoding apparatus (decoding method) of the subjectapplication. This makes it possible to improve coding efficiency at thetime when a prediction using a correlation within a screen is performed.Further, it is possible to decode an encoded stream having improvedcoding efficiency at the time when a prediction using a correlationwithin a screen is performed.

Eighth Embodiment

(Configuration Example of Imaging Apparatus)

FIG. 36 exemplifies a schematic configuration of an imaging apparatus towhich the present disclosure is applied. An imaging apparatus 960 imagesa subject and then displays an image of the subject on a display sectionor records such image data on a recording medium.

The imaging apparatus 960 includes an optical block 961, an imagingsection 962, a camera signal processing section 963, an image dataprocessing section 964, a display section 965, an external interfacesection 966, a memory section 967, a media drive 968, an OSD section969, and a control section 970. Further, a user interface section 971 isconnected to the control section 970. Moreover, the image dataprocessing section 964, the external interface section 966, the memorysection 967, the media drive 968, the OSD section 969, the controlsection 970, and the like are connected to one another via a bus 972.

The optical block 961 is constituted of a focus lens, a diaphragmmechanism, and the like. The optical block 961 forms an optical image ofthe subject on an imaging surface of the imaging section 962. Theimaging section 962 is constituted of a CCD or CMOS image sensor, andgenerates an electrical signal corresponding to the optical image byopto-electronic conversion and supplies the electrical signal to thecamera signal processing section 963.

The camera signal processing section 963 performs various types ofcamera signal processing, such as knee correction, gamma correction, andcolor correction, on the electrical signal supplied from the imagingsection 962. The camera signal processing section 963 supplies imagedata after subjected to the camera signal processing to the image dataprocessing section 964.

The image data processing section 964 performs encoding processing onthe image data supplied from the camera signal processing section 963.The image data processing section 964 supplies encoded data generated bythe encoding processing to the external interface section 966 or themedia drive 968. Further, the image data processing section 964 performsdecoding processing on encoded data supplied from the external interfacesection 966 or the media drive 968. The image data processing section964 supplies image data generated by the decoding processing to thedisplay section 965. Further, the image data processing section 964performs processing of supplying the image data, which is supplied fromthe camera signal processing section 963, to the display section 965, orsuperimposes display data acquired from the OSD section 969 on the imagedata and supplies such display data to the display section 965.

The OSD section 969 generates display data of a menu screen constitutedof symbols, characters, and graphics, icons, and the like and outputsthe display data to the image data processing section 964.

The external interface section 966 is constituted of, for example, a USBinput and output terminal and is connected to a printer in the casewhere an image is printed. Further, a drive is connected to the externalinterface section 966 as needed, and a removable medium such as amagnetic disk and an optical disc is appropriately mounted so as toinstall a computer program read therefrom as needed. Moreover, theexternal interface section 966 includes a network interface connected toa predetermined network such as a LAN (local area network) and theInternet. The control section 970 can read the encoded data from themedia drive 968, for example, according to an instruction from the userinterface section 971, and can supply the encoded data from the externalinterface section 966 to another apparatus connected via a network.Further, the control section 970 can acquire, via the external interfacesection 966, encoded data or image data supplied from another apparatusvia a network, and can supply the data to the image data processingsection 964.

As a recording medium driven by the media drive 968, for example, anarbitrary readable and writable removable medium such as a magneticdisk, a magneto-optical disk, an optical disc, and a semiconductormemory is used. Further, the recording medium may be of an arbitrarytype as a removable medium, and may be a tape device, a disc, or amemory card. As a matter of course, the recording medium may be anon-contact IC (Integrated Circuit) card or the like.

Further, the media drive 968 and the recording medium may be integratedand constituted of a non-transportable storage medium such as a built-inhard disk drive and an SSD (Solid State Drive).

The control section 970 is constituted of a CPU. The memory section 967stores programs executed by the control section 970, various types ofdata necessary for the control section 970 to perform processing, andthe like. The programs stored in the memory section 967 are read andexecuted by the control section 970 at a predetermined timing such asactivation of the imaging apparatus 960. The control section 970executes the programs to control the sections such that the imagingapparatus 960 operates according to a user operation.

In the imaging apparatus thus configured, the image data processingsection 964 is provided with a function of the encoding apparatus andthe decoding apparatus (encoding method and decoding method) of thesubject application. This makes it possible to improve coding efficiencyat the time when a prediction using a correlation within a screen isperformed. Further, it is possible to decode an encoded stream havingimproved coding efficiency at the time when a prediction using acorrelation within a screen is performed.

Ninth Embodiment Other Examples of Embodiment

The examples of the apparatuses, the systems, and the like to which thepresent disclosure is applied have been described hereinabove, but thepresent disclosure is not limited thereto and can be carried out as anyother configurations to be mounted to an apparatus constituting theapparatuses or systems as described above, for example, a processor as asystem LSI (Large Scale Integration) or the like, a module using aplurality of processors or the like, a unit using a plurality of modulesor the like, a set obtained by further adding another function to theunit, or the like (i.e., configuration of a part of an apparatus).

(Configuration Example of Video Set)

An example of the case where the present disclosure is carried out as aset will be described with reference to FIG. 37. FIG. 37 shows anexample of a schematic configuration of a video set to which the presentdisclosure is applied.

Multi-functionality of electronic apparatuses has been advanced inrecent years, and in the case where a part of a configuration thereof iscarried out by selling, provision, or the like in the development andmanufacturing thereof, there have been frequently found not only a casewhere the part is carried out as a configuration having one function butalso a case where the part is carried out as one set having a pluralityof functions by combining a plurality of configurations havingassociated functions.

A video set 1300 shown in FIG. 37 has such a multi-functionalizedconfiguration and is a combination of a device having a function onimage encoding and decoding (any one or both of them) with a devicehaving another function associated with that function.

As shown in FIG. 37, the video set 1300 includes a module groupincluding a video module 1311, an external memory 1312, a powermanagement module 1313, a front end module 1314, and the like, anddevices having associated functions, such as a connectivity 1321, acamera 1322, and a sensor 1323.

The module is assumed as a component in which several component-wisefunctions associated with one another are collected to have a cohesivefunction. A specific physical configuration is arbitrary, but theconfiguration is assumed to be, for example, one obtained by disposing aplurality of processors having respective functions, electronic circuitelements such as a resistor and a capacitor, other devices, and the likeon a wiring substrate or the like for integration. Further, it is alsoconceived that the module may be combined with another module, aprocessor, and the like, to be a new module.

In the case of the example of FIG. 37, the video module 1311 is acombination of configurations having functions on image processing andincludes an application processor, a video processor, a broadband modem1333, and an RF (Radio Frequency) module 1334.

The processor is one in which a configuration having a predeterminedfunction is integrated in a semiconductor chip by an SoC (System On aChip) and may be called, for example, a system LSI (Large ScaleIntegration). Such a configuration having a predetermined function maybe a logic circuit (hardware configuration), a CPU, a ROM, a RAM, or thelike, and a program (software configuration) executed by using thoseabove, or may be a combination of both the configurations. For example,the processor may include a logic circuit, a CPU, a ROM, a RAM, and thelike and may achieve a part of the functions by the logic circuit(hardware configuration) and the other functions by the program(software configuration) executed by the CPU.

The application processor 1331 of FIG. 37 is a processor that executesan application on image processing. The application executed by theapplication processor 1331 can not only perform computing processing butalso control configurations inside and outside the video module 1311,e.g., the video processor 1332, as needed in order to achieve apredetermined function.

The video processor 1332 is a processor having a function on imageencoding/decoding (one or both of them).

The broadband modem 1333 is a processor (or module) that performsprocessing on wired or wireless (or both of them) broadbandcommunication performed via broadband connections such as the Internetand a public telephone network. For example, the broadband modem 1333performs, for example, digital modulation on data to be sent (digitalsignal) for conversion into an analog signal or demodulates a receivedanalog signal for conversion into data (digital signal). For example,the broadband modem 1333 can perform digital modulation/demodulation onarbitrary information such as image data processed by the videoprocessor 1332, a stream in which image data is encoded, an applicationprogram, and setting data.

The RF module 1334 is a module that performs frequency conversion,modulation and demodulation, amplification, filter processing, or thelike on an RF (Radio Frequency) signal that is sent and received via anantenna. For example, the RF module 1334 performs frequency conversionor the like on a baseband signal generated by the broadband modem 1333and generates an RF signal. Further, for example, the RF module 1334performs frequency conversion or the like on an RF signal received viathe front end module 1314 and generates a baseband signal.

It should be noted that, as indicated by a dotted line 1341 in FIG. 37,the application processor 1331 and the video processor 1332 may beintegrated and configured as one processor.

The external memory 1312 is a module that is provided outside the videomodule 1311 and includes a storage device used by the video module 1311.The storage device of the external memory 1312 may be achieved by anyphysical configuration. In general, since the storage device isfrequently used for storing a large amount of data such as image data inunits of frame, the storage device may be desirably achieved by, forexample, a semiconductor memory that is relatively inexpensiveness andhas a large capacity, such as a DRAM (Dynamic Random Access Memory).

The power management module 1313 manages and controls power supply tothe video module 1311 (configurations within the video module 1311).

The front end module 1314 is a module that provides a front end function(a circuit at a sending and reception end on the antenna side) to the RFmodule 1334. As shown in FIG. 37, the front end module 1314 includes,for example, an antenna section 1351, a filter 1352, and anamplification section 1353.

The antenna section 1351 includes an antenna that sends and receives aradio signal, and peripheral configurations thereof. The antenna section1351 sends a signal supplied from the amplification section 1353 as aradio signal, and supplies a received radio signal to the filter 1352 asan electrical signal (RF signal). The filter 1352 performs filterprocessing or the like on the RF signal received via the antenna section1351 and supplies the processed RF signal to the RF module 1334. Theamplification section 1353 amplifies the RF signal supplied from the RFmodule 1334 and supplies the resultant signal to the antenna section1351.

The connectivity 1321 is a module having a function on externalconnection. A physical configuration of the connectivity 1321 isarbitrary. For example, the connectivity 1321 includes a configurationhaving a communication function other than communication standardscorresponding to the broadband modem 1333, an external input/outputterminal, and the like.

For example, the connectivity 1321 may include a module having acommunication function according to wireless communication standardssuch as Bluetooth (registered trademark), IEEE 802.11 (for example,Wi-Fi (Wireless Fidelity, registered trademark)), NFC (Near FieldCommunication), and IrDA (InfraRed Data Association), an antenna thatsends and receives a signal according to such standards, and the like.Further, for example, the connectivity 1321 may include a module havinga communication function according to wired communication standards suchas USB (Universal Serial Bus) and HDMI (registered trademark)(High-Definition Multimedia Interface), and a terminal according to suchstandards. Moreover, for example, the connectivity 1321 may have anotherdata (signal) transmission function or the like, such as an analoginput/output terminal.

It should be noted that the connectivity 1321 may include a device as adata (signal) transmission destination. For example, the connectivity1321 may include a drive (including not only a drive of a removablemedium but also a hard disk, an SSD (Solid State Drive), a NAS (NetworkAttached Storage), and the like) that reads and writes data on arecording medium such as a magnetic disk, an optical disc, amagneto-optical disk, and a semiconductor memory. Further, theconnectivity 1321 may include an image or audio output device (monitor,speaker, or the like).

The camera 1322 is a module having a function of imaging a subject andacquiring image data of the subject. The image data acquired by imagingof the camera 1322 is, for example, supplied to the video processor 1332and encoded.

The sensor 1323 is a module having an arbitrary sensor function, e.g.,an audio sensor, an ultrasonic sensor, an optical sensor, an illuminancesensor, an infrared sensor, an image sensor, a rotation sensor, anangular sensor, an angular velocity sensor, a velocity sensor, anacceleration sensor, an inclination sensor, a magnetic identificationsensor, a shock sensor, and a temperature sensor. Data detected by thesensor 1323 is, for example, supplied to the application processor 1331and used by an application or the like.

The configuration described above as a module may be achieved as aprocessor and, conversely, the configuration described as a processormay be achieved as a module.

In the video set 1300 configured as described above, the presentdisclosure can be applied to the video processor 1332 as will bedescribed later. Therefore, the video set 1300 can be carried out as aset to which the present disclosure is applied.

(Configuration Example of Video Processor)

FIG. 38 shows an example of a schematic configuration of the videoprocessor 1332 (FIG. 37) to which the present disclosure is applied.

In the case of the example of FIG. 38, the video processor 1332 has afunction of receiving input of a video signal and an audio signal andencoding those signals by a predetermined system and a function ofdecoding the encoded video data and audio data and reproducing andoutputting the video signal and the audio signal.

As shown in FIG. 38, the video processor 1332 includes a video inputprocessing section 1401, a first image zooming section 1402, a secondimage zooming section 1403, a video output processing section 1404, aframe memory 1405, and a memory control section 1406. Further, the videoprocessor 1332 includes an encoding/decoding engine 1407, video ES(Elementary Stream) buffers 1408A and 1408B, and audio ES buffers 1409Aand 1409B. Moreover, the video processor 1332 includes an audio encoder1410, an audio decoder 1411, a multiplexing section (MUX (Multiplexer))1412, a demultiplexing section (DMUX (Demultiplexer)) 1413, and a streambuffer 1414.

The video input processing section 1401 acquires a video signal inputfrom, e.g., the connectivity 1321 (FIG. 37), and converts the videosignal into digital image data. The first image zooming section 1402performs format conversion, image zooming processing, or the like on theimage data. The second image zooming section 1403 performs image zoomingprocessing on the image data in accordance with a format of adestination for output via the video output processing section 1404 orperform format conversion, image zooming processing, or the like similarto the first image zooming section 1402 on the image data. The videooutput processing section 1404 performs format conversion, conversioninto an analog signal, or the like on the image data and outputs thedata, as a reproduced video signal, to the connectivity 1321 (FIG. 37),for example.

The frame memory 1405 is a memory for image data shared with the videoinput processing section 1401, the first image zooming section 1402, thesecond image zooming section 1403, the video output processing section1404, and the encoding/decoding engine 1407. The frame memory 1405 isachieved as a semiconductor memory, e.g., a DRAM.

In response to a synchronization signal from the encoding/decodingengine 1407, the memory control section 1406 controls access for writeand read with respect to the frame memory 1405 according to a scheduleof access to the frame memory 1405, which is written in an accessmanagement table 1406A. The access management table 1406A is updated bythe memory control section 1406 in accordance with processing executedin the encoding/decoding engine 1407, the first image zooming section1402, the second image zooming section 1403, and the like.

The encoding/decoding engine 1407 performs encoding processing on imagedata and decoding processing on a video stream in which the image datais encoded data. For example, the encoding/decoding engine 1407 encodesimage data read from the frame memory 1405 and sequentially writes theimage data, as a video stream, in the video ES buffer 1408A. Further,for example, the encoding/decoding engine 1407 sequentially reads thevideo stream from the video ES buffer 1408B for decoding andsequentially writes the decoded video stream, as image data, in theframe memory 1405. In the encoding and decoding, the encoding/decodingengine 1407 uses the frame memory 1405 as a work area. Further, theencoding/decoding engine 1407 outputs a synchronization signal to thememory control section 1406 at a timing of start of processing on amacroblock basis, for example.

The video ES buffer 1408A buffers the video stream generated by theencoding/decoding engine 1407 and supplies the video stream to themultiplexing section (MUX) 1412. The video ES buffer 1408B buffers thevideo stream supplied from the demultiplexing section (DMUX) 1413 andsupplies the video stream to the encoding/decoding engine 1407.

The audio ES buffer 1409A buffers an audio stream generated by the audioencoder 1410 and supplies the audio stream to the multiplexing section(MUX) 1412. The audio ES buffer 1409B buffers the audio stream suppliedfrom the demultiplexing section (DMUX) 1413 and supplies the audiostream to the audio decoder 1411.

The audio encoder 1410 performs, e.g., digital conversion on an audiosignal input from, e.g., the connectivity 1321 (FIG. 37) and encodes theresultant signal by a predetermined system such as an MPEG audio systemand an AC3 (AudioCode number 3) system. The audio encoder 1410sequentially writes an audio stream, which is data in which the audiosignal is encoded, in the audio ES buffer 1409A. The audio decoder 1411decodes the audio stream supplied from the audio ES buffer 1409B,performs conversion into an analog signal, for example, and supplies theanalog signal, as a reproduced audio signal, to the connectivity 1321(FIG. 37), for example.

The multiplexing section (MUX) 1412 multiplexes the video stream and theaudio stream. The multiplexing method (i.e., the format of a bit streamgenerated by multiplexing) is arbitrary. Further, at the time ofmultiplexing, the multiplexing section (MUX) 1412 can also addpredetermined header information or the like to the bit stream. In otherwords, the multiplexing section (MUX) 1412 can convert the format of thestream by the multiplexing. For example, the multiplexing section (MUX)1412 multiplexes the video stream and the audio stream to convert thosestreams into a transport stream, which is a bit stream having a formatfor transfer. Further, for example, the multiplexing section (MUX) 1412multiplexes the video stream and the audio stream to convert thosestreams into data (file data) having a file format for recording.

The demultiplexing section (DMUX) 1413 demultiplexes the bit stream, inwhich the video stream and the audio stream are multiplexed, by a methodcorresponding to the multiplexing by the multiplexing section (MUX)1412. In other words, the demultiplexing section (DMUX) 1413 extractsthe video stream and the audio stream from the bit stream read from thestream buffer 1414 (separates the video stream and the audio stream fromeach other). In other words, the demultiplexing section (DMUX) 1413 canconvert the format of the stream by the demultiplexing (inverseconversion of the conversion by the multiplexing section (MUX) 1412).For example, the demultiplexing section (DMUX) 1413 can acquire thetransport stream supplied from, e.g., the connectivity 1321 or thebroadband modem 1333 (each of which is shown in FIG. 37) via the streambuffer 1414 and demultiplex the transport stream, to convert thetransport stream into the video stream and the audio stream. Further,for example, the demultiplexing section (DMUX) 1413 can acquire the filedata, which is read from various recording media by, e.g., theconnectivity 1321 (FIG. 37), via the stream buffer 1414 and demultiplexthe file data, to convert the file data into the video stream and theaudio stream.

The stream buffer 1414 buffers the bit stream. For example, the streambuffer 1414 buffers the transport stream supplied from the multiplexingsection (MUX) 1412 and supplies the transport stream to, e.g., theconnectivity 1321 or the broadband modem 1333 (each of which is shown inFIG. 37) at a predetermined timing or on the basis of a request from theoutside or the like.

Further, for example, the stream buffer 1414 buffers the file datasupplied from the multiplexing section (MUX) 1412, supplies the filedata to, e.g., the connectivity 1321 (FIG. 37) at a predetermined timingor on the basis of a request from the outside or the like, and recordsthe file data on various recording media.

Moreover, the stream buffer 1414 buffers the transport stream acquiredvia, e.g., the connectivity 1321 or the broadband modem 1333 (each ofwhich is shown in FIG. 37) and supplies the transport stream to thedemultiplexing section (DMUX) 1413 at a predetermined timing or on thebasis of a request from the outside or the like.

Further, the stream buffer 1414 buffers the file data read from variousrecording media in, e.g., the connectivity 1321 (FIG. 37) and suppliesthe file data to the demultiplexing section (DMUX) 1413 at apredetermined timing or on the basis of a request from the outside orthe like.

Next, an operation example of the video processor 1332 having such aconfiguration will be described. For example, a video signal that isinput to the video processor 1332 from the connectivity 1321 (FIG. 37)or the like is converted into digital image data of a predeterminedsystem such as a 4:2:2Y/Cb/Cr system in the video input processingsection 1401, and then sequentially written in the frame memory 1405.The digital image data is read by the first image zooming section 1402or the second image zooming section 1403, subjected to format conversioninto a predetermined system such as a 4:2:0Y/Cb/Cr system and zoomingprocessing, and written in the frame memory 1405 again. The image datais encoded by the encoding/decoding engine 1407 and written as a videostream in the video ES buffer 1408A.

Further, an audio signal that is input from the connectivity 1321 (FIG.37) or the like to the video processor 1332 is encoded by the audioencoder 1410 and written, as an audio stream, in the audio ES buffer1409A.

The video stream of the video ES buffer 1408A and the audio stream ofthe audio ES buffer 1409A are read and multiplexed by the multiplexingsection (MUX) 1412, and converted into a transport stream, file data, orthe like. The transport stream generated by the multiplexing section(MUX) 1412 is buffered by the stream buffer 1414, and then output to anexternal network via, e.g., the connectivity 1321 or the broadband modem1333 (each of which is shown in FIG. 37). Further, the file datagenerated by the multiplexing section (MUX) 1412 is buffered by thestream buffer 1414, and then output to the connectivity 1321 (FIG. 37)or the like and recorded on various recording media.

Further, the transport stream, which is input from the external networkto the video processor 1332 via, e.g., the connectivity 1321 or thebroadband modem 1333 (each of which is shown in FIG. 37), is buffered inthe stream buffer 1414 and then demultiplexed by the demultiplexingsection (DMUX) 1413. Further, the file data, which is read from variousrecording media in, e.g., the connectivity 1321 (FIG. 37) and input tothe video processor 1332, is buffered in the stream buffer 1414 and thendemultiplexed by the demultiplexing section (DMUX) 1413. In other words,the transport stream or the file data input to the video processor 1332is separated to be a video stream and an audio stream by thedemultiplexing section (DMUX) 1413.

The audio stream is supplied to the audio decoder 1411 via the audio ESbuffer 1409B and decoded, to reproduce an audio signal. Further, thevideo stream is written in the video ES buffer 1408B, and thensequentially read by the encoding/decoding engine 1407, decoded, andwritten in the frame memory 1405. The decoded image data is subjected tozooming processing by the second image zooming section 1403 and writtenin the frame memory 1405. The decoded image data is then read by thevideo output processing section 1404, subjected to format conversion ofa predetermined system such as a 4:2:2Y/Cb/Cr system, and converted intoan analog signal, to reproduce and output a video signal.

In the case where the present disclosure is applied to the videoprocessor 1332 configured as described above, the present disclosureaccording to each of the embodiments described above only needs to beapplied to the encoding/decoding engine 1407. In other words, forexample, the encoding/decoding engine 1407 only needs to have a functionof the encoding apparatus or the decoding apparatus according to thefirst to third embodiments. This enables the video processor 1332 toobtain an effect similar to that described above with reference to FIGS.1 to 31.

It should be noted that in the encoding/decoding engine 1407 the presentdisclosure (i.e., the function of the encoding apparatus or the decodingapparatus according to each of the embodiments described above) may beachieved by hardware such as a logic circuit, may be achieved bysoftware such as an embedded program, or may be achieved by both ofthem.

(Another Configuration Example of Video Processor)

FIG. 39 shows another example of a schematic configuration of the videoprocessor 1332 (FIG. 37) to which the present disclosure is applied. Inthe case of the example of FIG. 39, the video processor 1332 has afunction of encoding/decoding video data by a predetermined system.

More specifically, as shown in FIG. 39, the video processor 1332includes a control section 1511, a display interface 1512, a displayengine 1513, an image processing engine 1514, and an internal memory1515. Further, the video processor 1332 includes a codec engine 1516, amemory interface 1517, a multiplexing/demultiplexing section (MUX DMUX)1518, a network interface 1519, and a video interface 1520.

The control section 1511 controls operations of respective processingsections within the video processor 1332, such as the display interface1512, the display engine 1513, the image processing engine 1514, and thecodec engine 1516.

As shown in FIG. 39, the control section 1511 includes, for example, amain CPU 1531, a sub CPU 1532, and a system controller 1533. The mainCPU 1531 executes a program for controlling the operations of therespective processing sections within the video processor 1332, or thelike. The main CPU 1531 generates a control signal according to thatprogram or the like and supplies the control signal to the processingsections (i.e., controls the operations of the respective processingsections). The sub CPU 1532 plays an auxiliary role for the main CPU1531. For example, the sub CPU 1532 executes a child process, asubroutine, or the like of the program executed by the main CPU 1531 orthe like. The system controller 1533 controls operations of the main CPU1531 and the sub CPU 1532, such as specifying a program executed by themain CPU 1531 and the sub CPU 1532.

The display interface 1512 outputs image data to, e.g., the connectivity1321 (FIG. 37) or the like under the control of the control section1511. For example, the display interface 1512 converts the image data ofdigital data into an analog signal and outputs the analog signal as areproduced video signal or the image data of digital data withoutchange, to a monitoring apparatus of the connectivity 1321 (FIG. 37) orthe like.

The display engine 1513 performs, under the control of the controlsection 1511, various types of conversion processing such as formatconversion, size conversion, and color gamut conversion on the imagedata so as to be matched with hardware specifications of a monitoringapparatus or the like that displays an image of the image data.

The image processing engine 1514 performs predetermined image processingsuch as filter processing for, for example, image quality improvement,on the image data under the control of the control section 1511.

The internal memory 1515 is a memory that is provided inside the videoprocessor 1332 and shared with the display engine 1513, the imageprocessing engine 1514, and the codec engine 1516. The internal memory1515 is used for, for example, giving and receiving data between thedisplay engine 1513, the image processing engine 1514, and the codecengine 1516. For example, the internal memory 1515 stores data suppliedfrom the display engine 1513, the image processing engine 1514, or thecodec engine 1516 and supplies the data to the display engine 1513, theimage processing engine 1514, or the codec engine 1516 as needed (forexample, in response to a request). The internal memory 1515 may beachieved by any storage device. In general, since the internal memory1515 is frequently used for storing a small amount of data such as imagedata in units of block and parameters, the internal memory 1515 may bedesirably achieved by a semiconductor memory that has a relatively(compared with, for example, the external memory 1312) small capacitybut has a high speed response, such as an SRAM (Static Random AccessMemory).

The codec engine 1516 performs processing on encoding or decoding of theimage data. An encoding/decoding system corresponding to the codecengine 1516 is arbitrary, and the number thereof may be one or more thanone. For example, the codec engine 1516 may have codec functions of aplurality of encoding/decoding systems and perform encoding of imagedata or decoding of encoded data by a function selected from the abovefunctions.

In the example shown in FIG. 39, the codec engine 1516 includes, asfunctional blocks of processing on the codec, for example, an MPEG-2Video 1541, an AVC/H.264 1542, an HEVC/H.265 1543, an HEVC/H.265(Scalable) 1544, an HEVC/H.265 (Multi-view) 1545, and an MPEG-DASH 1551.

The MPEG-2 Video 1541 is a functional block for encoding and decodingthe image data by an MPEG-2 system. The AVC/H.264 1542 is a functionalblock for encoding and decoding the image data by an AVC system. TheHEVC/H.265 1543 is a functional block for encoding and decoding theimage data by an HEVC. The HEVC/H.265 (Scalable) 1544 is a functionalblock for scalable encoding and scalable decoding of the image data bythe HEVC. The HEVC/H.265 (Multi-view) 1545 is a functional block formulti-view encoding and multi-view decoding of the image data by theHEVC.

The MPEG-DASH 1551 is a functional block for sending and receiving theimage data by an MPEG-DASH (MPEG-Dynamic Adaptive Streaming over HTTP)system. The MPEG-DASH is a video streaming technique using an HTTP(HyperText Transfer Protocol), and one of the features thereof is toselect and transmit an appropriate one piece of encoded data in units ofsegment from pieces of the encoded data previously prepared that aredifferent in resolutions or the like. The MPEG-DASH 1551 performsgeneration of a stream according to standards, transmission and controlof the stream, and the like and uses the above-mentioned MPEG-2 Video1541 to the HEVC/H.265 (Multi-view) 1545 for encoding/decoding of theimage data.

The memory interface 1517 is an interface for the external memory 1312.Data supplied from the image processing engine 1514 or the codec engine1516 is supplied to the external memory 1312 via the memory interface1517. Further, data read from the external memory 1312 is supplied tothe video processor 1332 (the image processing engine 1514 or the codecengine 1516) via the memory interface 1517.

The multiplexing/demultiplexing section (MUX DMUX) 1518 multiplexes ordemultiplexes various types of data on images, such as a bit stream ofencoded data, image data, and a video signal. Themultiplexing/demultiplexing method is arbitrary. For example, at thetime of multiplexing, the multiplexing/demultiplexing section (MUX DMUX)1518 can not only collect pieces of data into one but also addpredetermined header information or the like to that data. Further, atthe time of demultiplexing, the multiplexing/demultiplexing section (MUXDMUX) 1518 can not only divide on piece of data into pieces of data butalso add predetermined header information or the like to the dividedpieces of data. In other words, the multiplexing/demultiplexing section(MUX DMUX) 1518 can convert a data format bymultiplexing/demultiplexing. For example, by multiplexing a bit stream,the multiplexing/demultiplexing section (MUX DMUX) 1518 can convert thebit stream into a transport stream, which is a bit stream having aformat for transfer, or data (file data) having a file format forrecording. As a matter of course, inverse conversion thereof can also beperformed by demultiplexing.

The network interface 1519 is an interface for, for example, thebroadband modem 1333, the connectivity 1321 (each of which is shown inFIG. 37), or the like. The video interface 1520 is an interface for, forexample, the connectivity 1321, the camera 1322 (each of which is shownin FIG. 37), or the like.

Next, an operation example of the video processor 1332 as describedabove will be described. For example, when a transport stream isreceived from an external network via the connectivity 1321, thebroadband modem 1333 (each of which is shown in FIG. 37), or the like,the transport stream is supplied to the multiplexing/demultiplexingsection (MUX DMUX) 1518 via the network interface 1519 anddemultiplexed, and then decoded by the codec engine 1516. Image dataobtained by the decoding of the codec engine 1516 is, for example,subjected to predetermined image processing by the image processingengine 1514, subjected to predetermined conversion by the display engine1513, and supplied to, e.g., the connectivity 1321 (FIG. 37) or the likevia the display interface 1512, so that an image thereof is displayed ona monitor. Further, for example, image data obtained by the decoding ofthe codec engine 1516 is encoded again by the codec engine 1516,multiplexed and converted into file data by themultiplexing/demultiplexing section (MUX DMUX) 1518, output to, e.g.,the connectivity 1321 (FIG. 37) or the like via the video interface1520, and recorded on various recording media.

Moreover, for example, file data of encoded data, in which image data isencoded and which is read from a recording medium not shown in thefigure by the connectivity 1321 (FIG. 37) or the like, is supplied tothe multiplexing/demultiplexing section (MUX DMUX) 1518 via the videointerface 1520 and demultiplexed, and then decoded by the codec engine1516. Image data obtained by the decoding of the codec engine 1516 issubjected to predetermined image processing by the image processingengine 1514, subjected to predetermined conversion by the display engine1513, and supplied to, e.g., the connectivity 1321 (FIG. 37) or the likevia the display interface 1512, so that an image thereof is displayed ona monitor. Further, for example, image data obtained by the decoding ofthe codec engine 1516 is encoded again by the codec engine 1516,multiplexed and converted into a transport stream by themultiplexing/demultiplexing section (MUX DMUX) 1518, supplied to, e.g.,the connectivity 1321, the broadband modem 1333 (each of which is shownin FIG. 37), or the like via the network interface 1519, and transmittedto another apparatus not shown in the figure.

It should be noted that giving and receiving of image data or other databetween the processing sections of the video processor 1332 is performedby using, for example, the internal memory 1515 or the external memory1312. Further, the power management module 1313 controls power supply tothe control section 1511, for example.

In the case where the present disclosure is applied to the videoprocessor 1332 configured as described above, the present disclosureaccording to each of the embodiments described above only needs to beapplied to the codec engine 1516. In other words, for example, the codecengine 1516 only needs to include a functional block that achieves theencoding apparatus or the decoding apparatus according to the first tothird embodiments. With the codec engine 1516 thus configured, the videoprocessor 1332 can obtain an effect similar to that described above withreference to FIGS. 1 to 31.

It should be noted that in the codec engine 1516 the present disclosure(i.e., the function of the encoding apparatus or the decoding apparatusaccording to each of the embodiments described above) may be achieved byhardware such as a logic circuit, may be achieved by software such as anembedded program, or may be achieved by both of them.

The two examples of the configuration of the video processor 1332 havebeen described above, but the configuration of the video processor 1332is arbitrary and may be other than the two examples described above.Further, the video processor 1332 may be configured as one semiconductorchip, but may be configured as a plurality of semiconductor chips. Forexample, the video processor 1332 may be a three-dimensional stacked LSIthat stacks a plurality of semiconductors. Further, the video processor1332 may be achieved by a plurality of LSIs.

(Example of Application to Apparatus)

The video set 1300 can be incorporated into various apparatuses thatprocess image data. For example, the video set 1300 can be incorporatedinto the television apparatus 900 (FIG. 33), the mobile phone 920 (FIG.34), the recording/reproduction apparatus 940 (FIG. 35), the imagingapparatus 960 (FIG. 36), or the like. By incorporating the video set1300 therein, the apparatus can obtain an effect similar to thatdescribed above with reference to FIGS. 1 to 31.

It should be noted that if a part of each configuration of the video set1300 described above includes the video processor 1332, the part can becarried out as a configuration to which the present disclosure isapplied. For example, only the video processor 1332 can be carried outas a video processor to which the present disclosure is applied.Further, for example, as described above, the processor indicated by thedotted line 1341, the video module 1311, or the like can be carried outas a processor, a module, or the like to which the present disclosure isapplied. Moreover, for example, it is also possible to combine the videomodule 1311, the external memory 1312, the power management module 1313,and the front end module 1314 to be carried out as a video unit 1361 towhich the present disclosure is applied. In the case of anyconfiguration, it is possible to obtain an effect similar to thatdescribed above with reference to FIGS. 1 to 31.

In other words, similarly to the case of the video set 1300, anyconfiguration including the video processor 1332 can be incorporated invarious apparatuses that process image data. For example, the videoprocessor 1332, the processor indicated by the dotted line 1341, thevideo module 1311, or the video unit 1361 can be incorporated in thetelevision apparatus 900 (FIG. 33), the mobile phone 920 (FIG. 34), therecording/reproduction apparatus 940 (FIG. 35), the imaging apparatus960 (FIG. 36), or the like. By incorporating any configuration to whichthe present disclosure is applied, such an apparatus can obtain aneffect similar to that described above with reference to FIGS. 1 to 31,similarly to the case of the video set 1300.

It should be noted that the example in which various types ofinformation are multiplexed into encoded data and transmitted from theencoding side to the decoding side has been described in thisspecification. However, the technique of transmitting those pieces ofinformation is not limited to such an example. For example, those piecesof information may be transmitted or recorded as individual pieces ofdata associated with encoded data, without being multiplexed into theencoded data. Here, the term “associated” means that an image includedin a bit stream (which may be a part of an image, such as slice andblock) and information corresponding to that image can be linked to eachother at the time of decoding. In other words, the information may betransmitted through a transmission path different from that for theencoded data. Further, the information may be recorded on a recordingmedium (or another recording area of the same recording medium)different from that for the encoded data. Moreover, the information andthe encoded data may be associated with each other in arbitrary unitsof, e.g., a plurality of frames, one frame, or a part of a frame.

The present disclosure can be applied to an encoding apparatus or adecoding apparatus that is used when, as in MPEG, H.26x, or the like, abit stream compressed by orthogonal transformation such as discretecosine transform and motion compensation is received via a networkmedium such as satellite broadcasting, a cable TV, the Internet, and amobile phone or is processed on a storage medium such as an opticaldisc, a magnetic disk, and a flash memory.

Further, in this specification, the system means an aggregation of aplurality of constituent elements (apparatus, module (component), andthe like), regardless of whether all constituent elements are includedin the same casing. Therefore, a plurality of apparatuses accommodatedin separate casings and connected to one another via a network is asystem, and one apparatus including a plurality of modules in one casingis also a system.

Moreover, the effects described in this specification are merelyexemplary ones and are not restrictive ones, and any other effects maybe produced.

Further, the embodiments of the present disclosure are not limited tothe embodiments described above and can be variously modified withoutdeparting from the gist of the present disclosure.

For example, in the first and second embodiments, the position of thepicture identifying information of the current picture registered in thereference picture list may not be fixed. Further, in the first and thirdembodiments, the picture type of the current picture set in thereference picture list may be STRP.

Further, in the second embodiment, as in the third embodiment, thepicture identifying information of the current picture may be registerednot in the reference picture list but in the temporal list.

Moreover, in the third embodiment, as in the second embodiment, by achange of the method of generating a predicted vector list in the casewhere the type of the reference picture of the current picture is IntraBC, coding efficiency in the case where Intra BC and inter coding arecommunalized may be improved.

Further, for example, the present disclosure can have a configuration ofcloud computing in which a plurality of apparatuses share one functionand cooperate to perform processing via a network.

Further, the steps described in the flowcharts described above can beexecuted by one apparatus or shared and executed by a plurality ofapparatuses.

Moreover, in the case where one step includes a plurality of processingsteps, the plurality of processing steps in one step can be executed byone apparatus or shared and executed by a plurality of apparatuses.

The present disclosure can also have the following configurations.

(1) An image processing apparatus, including:

a predicted vector generation section that sets, at a time of encodingof a current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the candidate block to notavailable and generates a predicted vector of the current motion vectorby using the reference motion vector, the reference motion vector beingreferred to when the predicted vector of the current motion vector isgenerated; and

a difference vector generation section that generates a differencevector between the current motion vector and the predicted vectorgenerated by the predicted vector generation section.

(2) The image processing apparatus according to (1), further including

a setting section that sets the type of the reference picture of thecurrent block to a long-term reference picture.

(3) The image processing apparatus according to (1), in which

the predicted vector generation section determines that the type of thereference picture of the current block and the type of the referencepicture of the candidate block are different from each other in a casewhere a picture of the candidate block and the reference picture of thecandidate block are different from each other.

(4) The image processing apparatus according to any one of (1) to (3),further including

a list creation section that registers a picture of the current block ina predetermined position of a list of candidates of the referencepicture of the current block.

(5) The image processing apparatus according to (4), in which

the predetermined position is a head.

(6) The image processing apparatus according to (4) or (5), in which

the list creation section rearranges order of the candidates of thereference picture in the list, before registering the picture of thecurrent block.

(7) The image processing apparatus according to (4) or (5), in which

the list creation section rearranges order of the candidates of thereference picture other than the picture of the current block in thelist, after registering the picture of the current block.

(8) An image processing method for an image processing apparatus, theimage processing method including:

a predicted vector generation step of setting, at a time of encoding ofa current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the candidate block to notavailable and generates a predicted vector of the current motion vectorby using the reference motion vector, the reference motion vector beingreferred to when the predicted vector of the current motion vector isgenerated; and

a difference vector generation step of generating a difference vectorbetween the current motion vector and the predicted vector generated byprocessing of the predicted vector generation step.

(9) An image processing apparatus, including:

a predicted vector generation section that sets, at a time of decodingof a current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the candidate block to notavailable and generates a predicted vector of the current motion vectorby using the reference motion vector, the reference motion vector beingreferred to when the predicted vector of the current motion vector isgenerated; and

a motion vector generation section that adds a difference vector betweenthe current motion vector and the predicted vector to the predictedvector generated by the predicted vector generation section andgenerates the current motion vector.

(10) The image processing apparatus according to (9), further including

a setting section that sets the type of the reference picture of thecurrent block to a long-term reference picture.

(11) The image processing apparatus according to (9), in which

the predicted vector generation section determines that the type of thereference picture of the current block and the type of the referencepicture of the candidate block are different from each other in a casewhere a picture of the candidate block and the reference picture of thecandidate block are different from each other.

(12) The image processing apparatus according to any one of (9) to (11),further including

a list creation section that registers a picture of the current block ina predetermined position of a list of candidates of the referencepicture of the current block.

(13) The image processing apparatus according to (12), in which

the predetermined position is a head.

(14) The image processing apparatus according to (12) or (13), in which

the list creation section rearranges order of the candidates of thereference picture in the list, before registering the picture of thecurrent block.

(15) The image processing apparatus according to (12) or (13), in which

the list creation section rearranges order of the candidates of thereference picture other than the picture of the current block in thelist, after registering the picture of the current block.

(16) An image processing method for an image processing apparatus, theimage processing method including:

a predicted vector generation step of setting, at a time of decoding ofa current motion vector of a current block for a prediction using acorrelation within a screen, in a case where a type of a referencepicture of the current block and a type of a reference picture of acandidate block corresponding to a candidate of a reference motionvector are different from each other, the candidate block to notavailable and generates a predicted vector of the current motion vectorby using the reference motion vector, the reference motion vector beingreferred to when the predicted vector of the current motion vector isgenerated; and

a motion vector generation step of adding a difference vector betweenthe current motion vector and the predicted vector to the predictedvector generated by processing of the predicted vector generation stepand generating the current motion vector.

REFERENCE SIGNS LIST

-   50 encoding apparatus-   88 list creation section-   90 predicted vector generation section-   91 difference vector generation section-   110 decoding apparatus-   144 predicted vector generation section-   145 motion vector generation section-   146 list creation section

1. An image processing apparatus, comprising: a reference motion vectorsetting section configured to exclude, in a case where one candidate ofa reference motion vector referred to when generating a current motionvector of a current block and a predicted vector of the current motionvector is a motion vector referring to Short-Term Reference Picture(STRP) and another candidate is a motion vector referring to same imageas a block being a reference source or Long-Term Reference Picture(LTRP), the one candidate from candidates of the reference motionvector; a predicted vector generation section configured to generate thepredicted vector by using the reference motion vector composed of thecandidates not excluded from candidates of the reference motion vector;and a decoding section configured to recreate the current motion vectorby using the generated predicted vector, wherein the reference motionvector setting section, the predicted vector generation section, and thedecoding section are each implemented via at least one processor.
 2. Theimage processing apparatus according to claim 1, wherein the referencemotion vector setting section is further configured to exclude, in acase where the current motion vector is the motion vector referring tocurrent picture and the one candidate is the motion vector referring tothe STRP, the one candidate from candidates of the reference motionvector.
 3. The image processing apparatus according to claim 1, whereinthe reference motion vector setting section is further configured toexclude, in a case where the current motion vector is the motion vectorreferring to the STRP and the one candidate is the motion vectorreferring to same image as the block being a reference source, the onecandidate from candidates of the reference motion vector.
 4. The imageprocessing apparatus according to claim 1, wherein the reference motionvector setting section is further configured to set, in a case where thecurrent motion vector is the motion vector referring to current pictureand the one candidate is the motion vector referring to same image asthe block being a reference source, the one candidate to the referencemotion vector.
 5. The image processing apparatus according to claim 4,wherein the predicted vector generation section is further configured toset the reference motion vector as the predicted vector.
 6. The imageprocessing apparatus according to claim 1, wherein the reference motionvector setting section is further configured to set, in a case where thecurrent motion vector is the motion vector referring to current pictureand the one candidate is the motion vector referring to the LTRP, theone candidate to the reference motion vector.
 7. The image processingapparatus according to claim 6, wherein the predicted vector generationsection is further configured to set the reference motion vector as thepredicted vector.
 8. The image processing apparatus according to claim1, wherein the reference motion vector setting section is furtherconfigured to set, in a case where the current motion vector is themotion vector referring to the LTRP and the one candidate is the motionvector referring to same image as the block being a reference source,the one candidate as the reference motion vector.
 9. The imageprocessing apparatus according to claim 8, wherein the predicted vectorgeneration section is further configured to set the reference motionvector as the predicted vector.
 10. The image processing apparatusaccording to claim 1, wherein the decoding section is further configuredto recreate the current motion vector by using a difference vectorbetween the generated predicted vector and the current motion vector andbetween the predicted vector and the current motion vector.
 11. An imageprocessing method, comprising: excluding, in a case where one candidateof a reference motion vector referred to when generating a currentmotion vector of a current block and a predicted vector of the currentmotion vector is a motion vector referring to Short-Term ReferencePicture (STRP) and another candidate is a motion vector referring tosame image as a block being a reference source or Long-Term ReferencePicture (LTRP), the one candidate from candidates of the referencemotion vector; generating the predicted vector by using the referencemotion vector composed of the candidates not excluded from candidates ofthe reference motion vector; and recreating the current motion vector byusing the generated predicted vector.
 12. The image processing methodaccording to claim 11, further comprising: excluding, in a case wherethe current motion vector is the motion vector referring to currentpicture and the one candidate is the motion vector referring to theSTRP, the one candidate from candidates of the reference motion vector.13. The image processing method according to claim 11, furthercomprising: excluding, in a case where the current motion vector is themotion vector referring to the STRP and the one candidate is the motionvector referring to same image as the block being a reference source,the one candidate from candidates of the reference motion vector. 14.The image processing method according to claim 11, further comprising:setting, in a case where the current motion vector is the motion vectorreferring to current picture and the one candidate is the motion vectorreferring to same image as the block being a reference source, the onecandidate to the reference motion vector.
 15. The image processingmethod according to claim 14, further comprising: setting the referencemotion vector as the predicted vector.
 16. The image processing methodaccording to claim 11, further comprising: setting, in a case where thecurrent motion vector is the motion vector referring to current pictureand the one candidate is the motion vector referring to the LTRP, theone candidate to the reference motion vector.
 17. The image processingmethod according to claim 16, further comprising: setting the referencemotion vector as the predicted vector.
 18. The image processing methodaccording to claim 11, further comprising: setting, in a case where thecurrent motion vector is the motion vector referring to the LTRP and theone candidate is the motion vector referring to same image as the blockbeing a reference source, the one candidate as the reference motionvector.
 19. The image processing method according to claim 18, furthercomprising: setting the reference motion vector as the predicted vector.20. The image processing method according to claim 11, furthercomprising: recreating the current motion vector by using a differencevector between the generated predicted vector and the current motionvector and between the predicted vector and the current motion vector.